{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[Artificial Intelligence for Humans](http://www.heatonresearch.com/aifh/)\n",
    "\n",
    "**Introduction to the Math of Neural Networks**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Understanding the Summation Operator\n",
    "\n",
    "You will frequently summations, as shown below:\n",
    "\n",
    "**Equation 1.1: The Summation Operator**\n",
    "$$ s = \\sum_{i=1}^{10} 2i $$\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you were to write the above equation as code (using Numpy/Python) you would have the following:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "110\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "\n",
    "i = np.arange(1,11)  # 11, because arange is not inclusive\n",
    "s = np.sum(2*i)\n",
    "print(s)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "More traditional looping (non-Numpy) would perform the summation as follows:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "110\n"
     ]
    }
   ],
   "source": [
    "s = 0\n",
    "for i in range(1,11):\n",
    "    s += 2*i\n",
    "    \n",
    "print(s)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Understanding the Product Operator\n",
    "\n",
    "**Equation 1.2: The Product Operator**\n",
    "$$ s = \\prod_{i=1}^{5} 2i $$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "3840\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "\n",
    "i = np.arange(1,6)  # 6, because arange is not inclusive\n",
    "s = np.prod(2*i)\n",
    "print(s)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "3840\n"
     ]
    }
   ],
   "source": [
    "s = 1\n",
    "for i in range(1,6): # 6, because arange is not inclusive\n",
    "    s *= 2*i\n",
    "    \n",
    "print(s)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Regression and Classification"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Transfer Functions\n",
    "\n",
    "## Linear Transfer Function\n",
    "\n",
    "**Equation 1.4: The Linear Transfer Function**\n",
    "$$ f(x) = x $$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAhgAAAF5CAYAAADdxFNyAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAAPYQAAD2EBqD+naQAAGUZJREFUeJzt3X+M5Hd93/HXG+MECMlZgp5Tg5oEQcCGELILRU4Bi/Cr\nEAEN2MFbSFRTSBxDgy6hgBuhpKgVhBBMIHekCU2c8GNztjBRI0GdArURRuB4F0wbHIJqmwThnGND\nDmp+FMynf8zY3jvux87ud+Y735nHQzrJ9/XM3NtfnXff+5zvzFRrLQAAXbpX3wMAAIvHggEAdM6C\nAQB0zoIBAHTOggEAdM6CAQB0zoIBAHTOggEAdM6CAQB0zoIBAHRukAtGVZ1RVe+sqtuq6mtVdX1V\nrfQ9FwAwcu++B5hUVZ2W5JokH0ryjCS3JXlYki/3ORcAcI8a2oedVdUbkpzdWjun71kAgGMb4lMk\nz05yXVVdVlWHqmqzql7S91AAwD2GuGA8JMkvJflskqcneXuSt1bVz/U6FQBwtyE+RfLNJNe21p64\n5djvJHlsa+1fHOP2D8joWo2bk3xjVnMCwAK4T5IfTnJla+32Se44uIs8k9yS5Iajjt2Q5HnHuf0z\nkrx7qhMBwGJ7YZL3THKHIS4Y1yR5+FHHHp7k88e5/c1J8q53vStnnnnmFMdaPPv27csll1zS9xiD\n4pztjPM2OedsZ5y3ydxwww150YtelIy/l05iiAvGJUmuqaqLk1yW5PFJXpLkpce5/TeS5Mwzz8zK\nirfKmMSePXucswk5ZzvjvE3OOdsZ523HJr7EYHAXebbWrkvyM0nWkvyvJL+W5BWttT/tdTAA4G5D\nLBhprb0/yfv7ngMAOLbBFQwAYP5ZMDiutbW1vkcYHOdsZ5y3yTlnO+O8zc7g3gdjUuMPQdvY2Nhw\nYQ8ATGBzczOrq6tJstpa25zkvgoGANA5CwYA0DkLBgDQOQsGANA5CwYA0DkLBgDQOQsGANA5CwYA\n0DkLBgDQOQsGANA5CwYA0DkLBgDQOQsGANA5CwYA0DkLBgDQOQsGANA5CwYA9Onb3+57gqmwYABA\nH1pLDh5MHvaw5Oab+56mcxYMAJi1W29NzjsvOf/85LGPTb7v+/qeqHP37nsAAFgarSWXXZa87GVJ\n1eifzzuv76mmQsEAgFnYWi2e/OTkr/5qYZeLRMEAgOlaomqxlYIBANOyZNViKwUDALq2pNViKwUD\nALq0xNViKwUDALqgWhxBwQCA3VItvouCAQA7pVocl4IBADuhWpyQggEAk1AttkXBAIDtUi22TcEA\ngJNRLSamYADAiagWO6JgAMCxqBa7omAAwNFUi11TMADgLqpFZxQMAEhUi44pGAAsN9ViKgZdMKrq\nNVX1nap6c9+zADBAqsXUDLZgVNXjkvxCkuv7ngWAgVEtpm6QBaOq7p/kXUlekuQfex4HgCFRLWZi\nkAtGkv1J/ry19uG+BwFgIFpLDh5MzjorufrqUbW4/PJk796+J1tIg1swqur8JI9JcnHfswAwEKrF\nzA3qGoyqenCStyR5amvtW5Pcd9++fdmzZ88Rx9bW1rK2ttbhhADMFddabNv6+nrW19ePOHb48OEd\nP1611nY708xU1XOTXJHkziQ1PnxKkjY+9r3tqP+gqlpJsrGxsZGVlZVZjgtAn269NbnoouS9703O\nPTfZv9/TIRPa3NzM6upqkqy21jYnue+gCkaSDyb5saOOXZrkhiRvOHq5AGAJqRZzYVALRmvtjiSf\n2Xqsqu5Icntr7YZ+pgJgbqgWc2NQC8ZxqBYAy061mDuDXzBaaz/V9wwA9Ei1mEuDXzAAWFKqxVwb\n3PtgAID3tZh/CgYAw6FaDIaCAcAwqBaDomAAMN9Ui0FSMACYX6rFYCkYAMwf1WLwFAwA5otqsRAU\nDADmg2qxUBQMAPqnWiwcBQOA/qgWC0vBAKAfqsVCUzAAmC3VYikoGADMjmqxNBQMAKZPtVg6CgYA\n06VaLCUFA4DpUC2WmoIBQPdUi6WnYADQHdWCMQUDgG6oFmyhYACwO6oFx6BgALBzqgXHoWAAMDnV\ngpNQMACYjGrBNigYAGyPasEEFAwATk61YEIKBgDHp1qwQwoGAMemWrALCgYAR1It6ICCAcA9VAs6\nomAAoFrQOQUDYNmpFkyBggGwrFQLpkjBAFhGqgVTpmAALBPVghlRMACWhWrBDCkYAItOtaAHCgbA\nIlMt6ImCAbCIVAt6pmAALBrVgjkwuAWjqi6uqmur6itVdaiq3ldVP9r3XAC9ay05eDA566zk6qtH\n1eLyy5O9e/uejCU0uAUjyROTvC3J45M8NcmpSf6iqu7b61QAfVItmDODuwajtfasrb+vqn+T5NYk\nq0k+2sdMAL1xrQVzaogF42inJWlJvtT3IAAzpVowxwZXMLaqqkryliQfba19pu95AGZCtWAAhl4w\nDiQ5K8n5fQ8CMBOqBQMx2IJRVb+b5FlJnthau+Vkt9+3b1/27NlzxLG1tbWsra1NaUKADqkWTNn6\n+nrW19ePOHb48OEdP1611nY708yNl4vnJjmntXbjSW67kmRjY2MjKysrM5kPoFO33ppcdFHy3vcm\n556b7N/vpafMxObmZlZXV5NktbW2Ocl9B1cwqupAkrUkz0lyR1WdPv5Xh1tr3+hvMoCOqRYM2BCv\nwbgwyQ8kuSrJF7f8+tkeZwLolmstGLjBFYzW2hCXIoDtUS1YEL5ZA8wL1YIFMriCAbBwVAsWkIIB\n0CfVggWlYAD0QbVgwSkYALOmWrAEFAyAWVEtWCIKBsAsqBYsGQUDYJpUC5aUggEwLaoFS0zBAOia\nagEKBkCnVAtIomAAdEO1gCMoGAC7pVrAd1EwAHZKtYDjUjAAdkK1gBNSMAAmoVrAtigYANulWsC2\nKRgAJ6NawMQUDIATUS1gRxQMgGNRLWBXFAyAo6kWsGsKBsBdVAvojIIBkKgW0DEFA1huqgVMhYIB\nLC/VAqZGwQCWj2oBU6dgAMtFtYCZUDCA5aBawEwpGMDiUy1g5hQMYHGpFtAbBQNYTKoF9ErBABaL\nagFzQcEAFodqAXNDwQCGT7WAuaNgAMOmWsBcUjCAYVItYK4pGMDwqBYw9xQMYDhUCxgMBQMYBtUC\nBkXBAOabagGDpGAA80u1gMEa7IJRVS+rqpuq6utV9fGqelzfMwEdaS05eDA566zk6qtH1eLyy5O9\ne/ueDNimQS4YVfWCJL+d5NeT/ESS65NcWVUP7HUwYPdUC1gIg1wwkuxL8l9aa3/SWvvrJBcm+VqS\nF/c7FrBjqgUslIkXjKr646p60jSG2eaff2qS1SQfuutYa60l+WCSs/uaC9gF1QIWzk4Kxp4kH6yq\nz1XVf6iqB3U91Ek8MMkpSQ4ddfxQkh+c8SzAbqgWsLAmXjBaa/8qyYOSvD3JC5LcXFUfqKpzx3UB\n4OQOHVItYIHt6H0wWmv/kOTNSd5cVStJLkjyziT/t6releRAa+1z3Y15hNuS3Jnk9KOOn57k7493\np3379mXPnj1HHFtbW8va2lrnAwIn4H0tYC6tr69nfX39iGOHDx/e8ePV6PKFHd656p8m+fmMFowH\nJ3lvRnXjnCSvaq1dsuMHP/Gf+/Ekn2itvWL8+0ryt0ne2lr7raNuu5JkY2NjIysrK9MYB9iuQ4eS\niy5KrrgiOffcZP9+T4fAHNvc3Mzq6mqSrLbWNie578QFY/w0yHMyWiqenuTTSd6S5D2tta+Mb7OW\n5ECSqSwYGdWTS6tqI8m1Gb2q5H5JLp3SnwfshmoBS2cnT5HcktG1G+tJ/nlr7VPHuM1/T/KPuxns\nRFprl43f8+J1GT018qkkzxg/dQPME9UCltJOFox9SS5vrX3jeDdorX05yY/seKptaK0dyKiSAPNI\ntYCltpNXkbzzRMsFQA4dGtUKrxCBpeXTVIHuqBbA2FDfKhyYN6oFsIWCAeyOagEcg4IB7JxqARyH\nggFMTrUATkLBACajWgDboGAA26NaABNQMICTUy2ACSkYwPGpFsAOKRjAsakWwC4oGMCRVAugAwoG\ncA/VAuiIggGoFkDnFAxYdqoFMAUKBiwr1QKYIgUDlpFqAUyZggHLRLUAZkTBgGWhWgAzpGDAolMt\ngB4oGLDIVAugJwoGLCLVAuiZggGLRrUA5oCCAYtCtQDmiIIBi0C1AOaMggFDploAc0rBgKFSLYA5\npmDA0KgWwAAoGDAkqgUwEAoGDIFqAQyMggHzTrUABkjBgHmlWgADpmDAPFItgIFTMGCeqBbAglAw\nYF6oFsACUTCgb6oFsIAUDOiTagEsKAUD+qBaAAtOwYBZUy2AJaBgwKyoFsASUTBgFlQLYMkMasGo\nqh+qqndU1Y1V9bWq+lxV/UZVndr3bHBMrSUHDyaPfGTykY+MqsXllyd79/Y9GcBUDe0pkkckqSQv\nTfJ/kjwqyTuS3C/Jq3qcC77boUPJRRclV1wxqhf791ssgKUxqAWjtXZlkiu3HLq5qt6U5MJYMJgX\nrrUAGNZTJMdxWpIv9T0EJHGtBcDYoArG0arqoUlenuRX+p6FJadaABxhLhaMqnp9klef4CYtyZmt\ntb/Zcp8HJflAkoOttT882Z+xb9++7Nmz54hja2trWVtb29nQcBfXWgALYH19Pevr60ccO3z48I4f\nr1pru51p16rqAUkecJKb3dha+/b49mck+Z9JPtZau+Akj72SZGNjYyMrKyudzAtJvrtaHDigWgAL\nZXNzM6urq0my2lrbnOS+c1EwWmu3J7l9O7cdl4sPJ/nLJC+e5lxwXKoFwAnNxYKxXeNycVWSmzJ6\n1cjeqkqStNYO9TcZS8O1FgDbMqgFI8nTkjxk/Ovvxscqo2s0TulrKJaEagGwbYN6mWpr7Y9ba6cc\n9eterTXLBdPj3TgBJjaoBQNmzvtaAOzI0J4igdlwrQXArigYcDTVAmDXFAy4i2oB0BkFAxLVAqBj\nCgbLTbUAmAoFg+WlWgBMjYLB8lEtAKZOwWC5qBYAM6FgsBxUC4CZUjBYfKoFwMwpGCwu1QKgNwoG\ni0m1AOiVgsFiUS0A5oKCweJQLQDmhoLB8KkWAHNHwWDYVAuAuaRgMEyqBcBcUzAYHtUCYO4pGAyH\nagEwGAoGw6BaAAyKgsF8Uy0ABknBYH6pFgCDpWAwf1QLgMFTMJgvqgXAQlAwmA+qBcBCUTDon2oB\nsHAUDPqjWgAsLAWDfqgWAAtNwWC2VAuApaBgMDuqBcDSUDCYPtUCYOkoGEyXagGwlBQMpkO1AFhq\nCgbdUy0Alp6CQXdUCwDGFAy6oVoAsIWCwe6oFgAcg4LBzqkWAByHgsHkVAsATmKwBaOqvqeqPlVV\n36mqR/c9z9JQLQDYhsEuGEnemOQLSVrfgyyF1pKDB5NHPjL5yEdG1eLyy5O9e/ueDIA5NMgFo6qe\nmeRpSV6ZpHoeZ/GpFgBMaHDXYFTV6Ul+P8lzkny953EWm2stANihIRaMP0pyoLX2yb4HWWiqBQC7\nMBcFo6pen+TVJ7hJS3Jmkn+Z5P5JfvOuu055tOWjWgDQgblYMJK8KaMycSI3JXlykrOTfLPqiN3i\nuqp6d2vtguPded++fdmzZ88Rx9bW1rK2traziRfRoUPJRRclV1wxqhf797uIE2BJrK+vZ319/Yhj\nhw8f3vHjVWvDeRFGVT04yQ9sOXRGkiuTPD/Jta21Lx7jPitJNjY2NrKysjKbQYfm6Gpx4IBqAUA2\nNzezurqaJKuttc1J7jsvBWNbWmtf2Pr7qrojo6dJbjzWcsE2qBYATMGgFozjGE6CmSeutQBgiob4\nKpK7tdY+31o7pbX26b5nGRSvEAFgyhahYLBdqgUAMzLogsEEVAsAZkjBWHSqBQA9UDAWmWoBQE8U\njEWkWgDQMwVj0agWAMwBBWNRqBYAzBEFYxGoFgDMGQVjyFQLAOaUgjFUqgUAc0zBGBrVAoABUDCG\nRLUAYCAUjCFQLQAYGAVj3qkWAAyQgjGvVAsABkzBmEeqBQADp2DME9UCgAWhYMwL1QKABaJg9E21\nAGABKRh9Ui0AWFAKRl+++tXkx388ufNO1QKAhWPB6Mv3f39yySXJU56S7N3b9zQA0CkLRp/W1vqe\nAACmwjUYAEDnLBgAQOcsGABA5ywYAEDnLBgAQOcsGABA5ywYAEDnLBgAQOcsGABA5ywYAEDnLBgA\nQOcsGABA5ywYAEDnLBgAQOcsGABA5ywYAEDnLBgAQOcsGABA5wa3YFTVT1fVx6vqa1X1paq6ou+Z\nFtX6+nrfIwyOc7YzztvknLOdcd5mZ1ALRlU9P8mfJPmvSX4syU8meU+vQy0w/yNOzjnbGedtcs7Z\nzjhvs3PvvgfYrqo6Jclbkvxqa+3SLf/qr/uZCAA4niEVjJUkZyRJVW1W1Rer6v1V9cie5wIAjjKk\nBeMhSSrJryd5XZKfTvLlJFdV1Wl9DgYAHKn3p0iq6vVJXn2Cm7QkZ+aeZeg/tdb+bHzfC5J8Icl5\nSf7gOPe/T5LccMMNncy7TA4fPpzNzc2+xxgU52xnnLfJOWc747xNZsv3zvtMet9qrXU7zaQDVD0g\nyQNOcrMbkzwhyYeTPKG19rEt9/94kv/RWnvtcR7/Xyd5d0fjAsAyemFrbaIXVfReMFprtye5/WS3\nq6qNJN9M8vAkHxsfOzXJDyf5/AnuemWSFya5Ock3djctACyV+2T0ffbKSe/Ye8GYRFVdkuT5Sf5t\nRkvFqzK6FuMRrbXDfc4GANyj94IxoVcm+VZG74Vx3ySfSPJTlgsAmC+DKhgAwDAM6WWqAMBAWDAA\ngM4t3YLhw9J2rqq+p6o+VVXfqapH9z3PvKqqH6qqd1TVjeO/Z5+rqt8Yv+qJLarqZVV1U1V9ffz/\n5eP6nmmeVdXFVXVtVX2lqg5V1fuq6kf7nmtIquo1469hb+57lnlXVWdU1Tur6rbx17Lrq2plu/df\nqgXDh6Xt2hszemMzF+6c2CMyetfZlyY5K8m+JBcm+c99DjVvquoFSX47o3fn/Ykk1ye5sqoe2Otg\n8+2JSd6W5PFJnprk1CR/UVX37XWqgRgvsL+Q0d81TmD8DtnXZPT2EM/I6A0vfzWjd9De3mMsy0We\n4w9LuznJa4/6sDS2oaqemeRNGb1M+DNJHtNa+3S/Uw1HVb0yyYWttYf2Pcu8GL9J3idaa68Y/76S\n/F2St7bW3tjrcAMxXsZuTfKk1tpH+55nnlXV/ZNsJPmlJK9N8snW2q/0O9X8qqo3JDm7tXbOTh9j\nmQqGD0vboao6PcnvJ3lRkq/3PM5QnZbkS30PMS/GTxetJvnQXcfa6KedDyY5u6+5Bui0jIqiv1sn\ntz/Jn7fWPtz3IAPx7CTXVdVl46fjNqvqJZM8wDItGD4sbef+KMmB1ton+x5kiKrqoUlenuT3+p5l\njjwwySlJDh11/FCSH5z9OMMzLj5vSfLR1tpn+p5nnlXV+Ukek+TivmcZkIdkVHs+m+TpSd6e5K1V\n9XPbfYDBLxhV9frxBTvH+3Xn+CKoIz4sbfzN8oKMtv/zevsP6Ml2z1tV/XKS+yf5zbvu2uPYvZrg\n79rW+zwoyQeSHGyt/WE/k7OgDmR0jc/5fQ8yz6rqwRktYi9srX2r73kG5F5JNlprr22tXd9a+4OM\nPlT0wu0+wNDeyfNY3pTRT9gncmPGT48kufuj4Vpr/6+qbkzyz6Y02zzbznm7KcmTM0rW3xz9wHS3\n66rq3a21C6Y03zza7t+1JKMrsDP6gL6PttZ+cZqDDdBtSe5McvpRx09P8vezH2dYqup3kzwryRNb\na7f0Pc+cW03yT5Js1j1fxE5J8qSqenmS723LcjHiZG7Jlu+XYzcked52H2DwC8YMPixtIU1w3v5d\nkl/bcuiMjD705meTXDud6ebTds9Zcne5+HCSv0zy4mnONUSttW+N/598SpL/ltyd/J+S5K19zjbv\nxsvFc5Oc01r7277nGYAPZvSqwa0uzeib5RssF8d1TUbfL7d6eCb4fjn4BWO7WmtfrarfS/Ifq+oL\nuefD0lqSy3sdbo611r6w9fdVdUdGT5Pc2Fr7Yj9TzbdxubgqowL0qiR77/rBqbV29DUHy+zNSS4d\nLxrXZvRy3vtl9MWfY6iqA0nWkjwnyR3jC7CT5HBrzadFH0Nr7Y6MXvl2t/HXsdtba0f/hM49Lkly\nTVVdnOSyjF4a/ZKMXn6/LUuzYIz5sLRu2PhP7GkZXSD1kIxedpmMlrKWUZolSWvtsvHLLF+X0VMj\nn0ryjNbaP/Q72Vy7MKO/R1cddfyCjL6usT2+hp1Ea+26qvqZJG/I6GW9NyV5RWvtT7f7GEvzPhgA\nwOwM/lUkAMD8sWAAAJ2zYAAAnbNgAACds2AAAJ2zYAAAnbNgAACds2AAAJ2zYAAAnbNgAACds2AA\nAJ2zYAAzV1UPrKpbquo1W479ZFV9s6qe3OdsQDd82BnQi6p6ZpI/S3J2kr/J6NNU39da+/e9DgZ0\nwoIB9Kaq3pbRx9tfl+RRSR7XWvtWv1MBXbBgAL2pqvsk+d9JHpxkpbX2mZ5HAjriGgygTw9NckZG\nX4t+pOdZgA4pGEAvqurUJNcm+WSSzybZl+RRrbXbeh0M6IQFA+hFVf1WkucleXSSryW5KslXWmvP\n7nMuoBueIgFmrqrOSfLLSV7UWrujjX7S+fkkT6iqX+x3OqALCgYA0DkFAwDonAUDAOicBQMA6JwF\nAwDonAUDAOicBQMA6JwFAwDonAUDAOicBQMA6JwFAwDonAUDAOicBQMA6Nz/BwNgmtySZA2ZAAAA\nAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x192ced79668>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "def linear_transfer(t):\n",
    "    return t\n",
    "\n",
    "x = np.arange(-5.0, 5.0, 0.02)\n",
    "\n",
    "plt.xlabel(\"x\")\n",
    "plt.ylabel(\"y\")\n",
    "plt.plot(x, linear_transfer(x), 'r')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Softmax Transfer Function\n",
    "\n",
    "**Equation 1.4: The Softmax Transfer Function**\n",
    "$$ \\sigma(\\mathbf{z})_j = \\frac{e^{z_j}}{\\sum_{k=1}^K e^{z_k}} $$\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Rectifier Linear Unit (ReLU) Transfer Function\n",
    "\n",
    "** Equation 1.4: The ReLU Transfer Function **\n",
    "$$ f(x) = \\max(0, x) $$\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAhgAAAF5CAYAAADdxFNyAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAAPYQAAD2EBqD+naQAAF+NJREFUeJzt3XuQZGd5HvDnRVKQwdRSQIlwcbAJxojCBna4J4LCXIQd\n7ljgMQq3ioOAyK5NsIPJBYcq2wSDREARxJUKd0+BqRBDKrA2ApkCLJTMIIiLJRaFZLBRZC7K4hLg\nAunLH91rdtd7me75Zk6f7t+vako1Z/qM3rPdPfPM6e/pU621AAD0dJuhBwAAlo+AAQB0J2AAAN0J\nGABAdwIGANCdgAEAdCdgAADdCRgAQHcCBgDQnYABAHQ3uoBRVRdV1Wer6vD041NV9aSh5wIAfqDG\ndi2SqvpHSW5Jcm2SSvKCJL+S5EGttUMDjgYATI0uYJxIVX0jyctba28dehYAIDlz6AF2oqpuk+TZ\nSW6X5I8HHgcAmBplwKiqB2QSKM5O8ldJntFa+8KwUwEAR4zyJZKqOjPJ30uyL8nPJfnFJI8+Ucio\nqjsnOT/J9Um+u4djAsDYnZ3kR5McbK19Y5YdRxkwjldVf5jki621l5zga7+Q5N17PxUALI3nttZ+\nd5YdRvkSyQncJsltT/K165PkXe96V84999w9G2g3HThwIJdeeunQY3SxTMeSOJ5FtkzHkjiehXP1\n1cnLXpZccEEO3HDDuI/lKIcOHcqFF16YTH+XzmJ0AaOqfjPJh5J8Ockdkjw3yWOSPPEku3w3Sc49\n99zs379/T2bcbfv27XMsC8rxLK5lOpbE8SyUa69NXvnK5PGPT971rux75jPHeywnN/MSg9EFjCTn\nJHl7krslOZzkc0me2Fr76KBTAbB6bropefKTk3POSd7znuTMMf5a3R2j+5dorf2ToWcAgHz/+8mz\nn5187WvJpz+d3PGOQ0+0UEYXMABgIRw4kFx5ZXLwYPLjPz70NAtHwBih9fX1oUfoZpmOJXE8i2yZ\njiVxPIN785uTyy5L3vKW5Kd/+pgvje5YdslS1FRPpar2J9nc3NxcxkU3AOy1K65Izj8/eelLkze+\ncehpdtXW1lbW1taSZK21tjXLvqO7mioADObaa5MLLkge97jkkkuGnmahCRgAsB0aIzPxrwMAp6Mx\nMjMBAwBOR2NkZgIGAJzKKRojnJw1GABwMldckVx88eTjxS8eeppRETAA4EQ0RnZEwACA42mM7Jh/\nMQA4msZIFwIGABxNY6QLAQMAjtAY6cYaDABINEY6EzAAQGOkOwEDgNWmMbIr/CsCsLo0RnaNgAHA\n6tIY2TUCBgCrSWNkV1mDAcDq0RjZdQIGAKtFY2RPCBgArA6NkT3jXxaA1aAxsqcEDABWg8bInhIw\nAFh+GiN7zhoMAJabxsggBAwAlpfGyGAEDACWk8bIoPxrA7B8NEYGJ2AAsHw0RgYnYACwXDRGFoI1\nGAAsD42RhSFgALAcNEYWioABwPhpjCwc9wAA46YxspAEDADGTWNkIQkYAIyXxsjCsgYDgHHSGFlo\nAgYA46MxsvAEDADGRWNkFNwrAIyHxshoCBgAjIfGyGgIGACMg8bIqFiDAcDi0xgZHQEDgMWmMTJK\nAgYAi0tjZLTcUwAsJo2RURMwAFhMGiOjJmAAsHg0RkbPGgwAFovGyFIYXcCoql+rqqur6ltVdWNV\nvb+q7jv0XAB0oDGyNEYXMJKcl+RNSR6e5PFJzkryB1X1Q4NOBcDOaIwsldHde621nz3686p6QZK/\nTLKW5BNDzATADmmMLJ3RBYwTuGOSluSbQw8CwJw0RpbOqANGVVWSNyT5RGvt80PPA8AcNEaW0qgD\nRpLLk9w/yT8YehAA5qAxsrRGGzCq6rIkP5vkvNbaDae7/YEDB7Jv375jtq2vr2d9fX2XJgTglDRG\nFsrGxkY2NjaO2Xb48OG5v1+11nY6056bhounJXlMa+1Lp7nt/iSbm5ub2b9//57MB8Bp3HRT8ohH\nJFXJVVdZ1Lmgtra2sra2liRrrbWtWfYd3RmMqro8yXqSpya5uaruOv3S4dbad4ebDIBt0RhZCaML\nGEkuyqQ1cuVx21+Y5B17Pg0As9EYWQmjCxittTG+ORgAicbICvHLGoC9oTGyUgQMAHafxsjKETAA\n2F2uMbKS3MsA7B6NkZUlYACwezRGVpaAAcDu0BhZadZgANCfxsjKEzAA6EtjhAgYAPSkMcKUex6A\nPjRGOIqAAUAfGiMcRcAAYOc0RjiONRgA7IzGCCcgYAAwP40RTkLAAGA+GiOcgkcDALPTGOE0BAwA\nZqcxwmkIGADMRmOEbbAGA4Dt0xhhmwQMALZHY4QZCBgAnJ7GCDPyCAHg1DRGmIOAAcCpaYwwBwED\ngJPTGGFO1mAAcGIaI+yAgAHA36Yxwg4JGAAcS2OEDjxqAPgBjRE6ETAA+AGNEToRMACY0BihI2sw\nANAYoTsBA2DVaYywCwQMgFWmMcIu8UgCWFUaI+wiAQNgVWmMsIsEDIBVpDHCLrMGA2DVaIywBwQM\ngFWiMcIeETAAVoXGCHvIowtgFWiMsMcEDIBVoDHCHhMwAJadxggDsAYDYJlpjDAQAQNgWWmMMCAB\nA2AZaYwwMI84gGWjMcICEDAAlo3GCAtAwABYJhojLAhrMACWhcYIC0TAAFgGGiMsGAEDYOw0RlhA\nowsYVXVeVX2gqv6iqm6tqqcOPRPAYI5ujHzwgxojLIzRBYwkt09yTZKXJmkDzwIwrCONkfe9T2OE\nhTK682ittQ8n+XCSVFUNPA7AcDRGWGBjPIMBgMYIC07AABgbjRFGQMAAGBONEUZiZR6ZBw4cyL59\n+47Ztr6+nvX19YEmApiRa4ywizY2NrKxsXHMtsOHD8/9/aq18RYxqurWJE9vrX3gFLfZn2Rzc3Mz\n+/fv37vhAHq7+OLJgs6DBy3qZE9sbW1lbW0tSdZaa1uz7Du6MxhVdfsk90lypEFy76p6YJJvtta+\nMtxkALtIY4SRGV3ASPKQJB/L5D0wWpLXT7e/PcmLhhoKYNdojDBCowsYrbU/isWpwKrQGGGk/KIG\nWFQaI4yYRyvAItIYYeQEDIBFdOQaIwcPusYIoyRgACwajRGWgDUYAItEY4QlIWAALAqNEZaIgAGw\nCDRGWDIewQBD0xhhCQkYAEPTGGEJCRgAQ9IYYUlZgwEwFI0RlpiAATAEjRGWnIABsNc0RlgBHtUA\ne0ljhBUhYADsJY0RVoSAAbBXNEZYIdZgAOwFjRFWjIABsNs0RlhBAgbAbtIYYUV5pAPsFo0RVpiA\nAbBbNEZYYQIGwG7QGGHFWYMB0JvGCMweMKrq7VX16N0YBmD0NEYgyXxnMPYl+UhVXVtVr6yqe/Qe\nCmCUNEbgb8wcMFprT09yjyRvTvKcJNdX1Yeq6ueq6qzeAwKMwtGNkQ9+UGOElTfXGozW2tdaa5e0\n1h6Y5OFJvpjknUm+WlWXVpXl0sBqOdIYed/7NEYgO1zkWVV3S/KE6cctSf5Hkp9M8vmqOrDz8QBG\n4Ehj5LLLNEZgap5FnmdV1bOq6r8n+bMkFyR5Q5K7t9ae31p7fJLnJfm3fUcFWEAaI3BC86xAuiGT\nYLKR5GGttWtOcJsPJ/l/OxkMYOFpjMBJzRMwDiT5vdbad092g9baTUl+bO6pABadxgic0szPiNba\nO3djEIDRcI0ROC2RG2BWrjECpyVgAMzCNUZgW1yLBGC7NEZg2wQMgO3QGIGZCBgAp6MxAjPzLAE4\nFY0RmIuAAXAqGiMwFwED4GQ0RmBu1mAAnIjGCOyIgAFwPI0R2DEBA+BoGiPQhWcOwBEaI9CNgAFw\nhMYIdCNgACQaI9CZNRgAGiPQnYABrDaNEdgVAgawujRGYNd4NgGrSWMEdpWAAawmjRHYVaN9iaSq\nXlZV11XVd6rqqqp66NAzASNxpDFy2WUaI7BLRhkwquo5SV6f5FVJHpzks0kOVtVdBh0MWHwaI7An\nRhkwkhxI8p9aa+9orX0hyUVJvp3kRcOOBSw0jRHYM6MLGFV1VpK1JFcc2dZaa0k+kuSRQ80FLDiN\nEdhTY3yG3SXJGUluPG77jUl+4qR7HTq0iyMBC+8Vr9AYgT00xoAxnwsvHHoCYEhnnqkxAntojAHj\n60luSXLX47bfNcn/PdlOBx784Oy7wx2O2bZ+/vlZf9KTug8ILKBzzknuec+hp4CFtbGxkY2NjWO2\nHT58eO7vV5PlC+NSVVcl+XRr7Zenn1eSLyd5Y2vtt4+77f4km5ubm9m/f//eDwsAI7W1tZW1tbUk\nWWutbc2y7xjPYCTJJUneVlWbSa7OpFVyuyRvG3IoAGBilAGjtfbe6XtevDqTl0auSXJ+a+1rw04G\nACQjDRhJ0lq7PMnlQ88BAPxto3sfDABg8QkYAEB3AgYA0J2AAQB0J2AAAN0JGABAdwIGANCdgAEA\ndCdgAADdCRgAQHcCBgDQnYABAHQnYAAA3QkYAEB3AgYA0J2AAQB0J2AAAN0JGABAdwIGANCdgAEA\ndCdgAADdCRgAQHcCBgDQnYABAHQnYAAA3QkYAEB3AgYA0J2AAQB0J2AAAN0JGABAdwIGANCdgAEA\ndCdgAADdCRgAQHcCBgDQnYABAHQnYAAA3QkYAEB3AgYA0J2AAQB0J2AAAN0JGABAdwIGANCdgAEA\ndCdgAADdCRgAQHcCBgDQnYABAHQnYAAA3QkYAEB3AgYA0N2oAkZVvbKqPllVN1fVN4eeBwA4sVEF\njCRnJXlvkjcPPQgAcHJnDj3ALFpr/y5Jqur5Q88CAJzc2M5gAAAjIGAAAN0NHjCq6req6tZTfNxS\nVfcdek4AYPsWYQ3G65K89TS3+dJO/ycHDhzIvn37jtm2vr6e9fX1nX5rABi9jY2NbGxsHLPt8OHD\nc3+/aq3tdKY9N13keWlr7U7buO3+JJubm5vZv3//7g8HAEtia2sra2trSbLWWtuaZd9FOIOxbVX1\nI0nulOReSc6oqgdOv/TF1trNw00GABxtVAEjyauTPO+oz4+kqccm+fjejwMAnMjgizxn0Vp7YWvt\njBN8CBcAsEBGFTAAgHEQMACA7gQMAKA7AQMA6E7AAAC6EzAAgO4EDACgOwEDAOhOwAAAuhMwAIDu\nBAwAoDsBAwDoTsAAALoTMACA7gQMAKA7AQMA6E7AAAC6EzAAgO4EDACgOwEDAOhOwAAAuhMwAIDu\nBAwAoDsBAwDoTsAAALoTMACA7gQMAKA7AQMA6E7AAAC6EzAAgO4EDACgOwEDAOhOwAAAuhMwAIDu\nBAwAoDsBAwDoTsAAALoTMACA7gQMAKA7AQMA6E7AAAC6EzAAgO4EDACgOwEDAOhOwAAAuhMwAIDu\nBAwAoDsBAwDoTsAAALoTMACA7gQMAKC7UQWMqrpXVf3nqvpSVX27qq6tql+vqrOGnm0vbWxsDD1C\nN8t0LInjWWTLdCyJ41lky3QsOzGqgJHkfkkqyS8muX+SA0kuSvIbQw6115bpwbtMx5I4nkW2TMeS\nOJ5FtkzHshNnDj3ALFprB5McPGrT9VX1ukxCxq8OMxUAcLyxncE4kTsm+ebQQwAAPzDqgFFV90ny\nz5K8ZehZAIAfWIiXSKrqt5L8y1PcpCU5t7X2p0ftc48kH0ryntbafznFvmcnyaFDh3qMuhAOHz6c\nra2tocfoYpmOJXE8i2yZjiVxPItsmY7lqN+dZ8+6b7XW+k4zh6q6c5I7n+ZmX2qtfX96+7sn+ViS\nT7XWXnia7/0LSd7dZVAAWE3Pba397iw7LETAmMX0zMVHk/zPJP+4neYApuHl/CTXJ/nurg8IAMvj\n7CQ/muRga+0bs+w4qoAxPXPxR0muS/KCJLcc+Vpr7caBxgIAjrMQazBm8IQk955+fGW6rTJZo3HG\nUEMBAMca1RkMAGAcRl1TBQAWk4ABAHS3UgGjqq6vqluP+rilqkb/FuNV9Xeq6prpMf3U0PPMq6p+\nv6r+rKq+U1Vfrap3VNXdhp5rVst4Ub6qemVVfbKqbq6q0b1zblW9rKqumz62rqqqhw490zyq6ryq\n+kBV/cX0+f7UoWeaV1X9WlVdXVXfqqobq+r9VXXfoeeaV1VdVFWfrarD049PVdWThp6rh6p6xfTx\ndsks+61UwMhkMei/TnLXJH83yd2SvGnQifp4bZI/z+T4xuyjSS5Ict8kz0zy95P83qATzWcZL8p3\nVpL3Jnnz0IPMqqqek+T1SV6V5MFJPpvkYFXdZdDB5nP7JNckeWnG/3w/L5Ofvw9P8vhMHmN/UFU/\nNOhU8/tKJm8YuT/JWiY/z36/qs4ddKodmobxf5rJ82a2fVdpkWdVXZfk0tbaG4eepZeq+pkkr0vy\nrCSfT/Kg1trnhp2qj6p6SpL3J7lta+2W091+kVXVy5Nc1Fq7z9Cz7ERVPT+T59Cdhp5lu6rqqiSf\nbq398vTzyuSXwRtba68ddLgdqKpbkzy9tfaBoWfpYRr4/jLJo1trnxh6nh6q6htJXt5ae+vQs8yj\nqn44yWaSlyT5N0k+01r759vdf9XOYCTJK6rq61W1VVUvr6rR1lur6q5JfifJhUm+M/A4XVXVnZI8\nN8knxx4uplyUbwDTl6XWklxxZNv0zfk+kuSRQ83FCd0xk7Myo3+eVNVtqurnk9wuyR8PPc8O/Mck\nH2ytfXSencf2Phg79R+SbGXyAH5Uktdk8lLJy4ccagfemuTy1tpnqupeQw/TQ1W9JpML2B15Yj55\n2Il27qiL8m07+dPNXTJ5j5zj34jvxiQ/sffjcCLTs0pvSPKJ1trnh55nXlX1gEx+bp2d5K+SPKO1\n9oVhp5rPNCA9KMlD5v0eoz+DUVW/ddzCzeM/bjmycKi19obW2sdba3/SWvudJP8iycWLtPhuu8dT\nVb+U5IeT/Psjuw449knNcv9MvTaTB/UTMnmn1ncOMvgJzHEss1yUb8/NczywSy7PZL3Szw89yA59\nIckDkzwsk/VK76iq+w070uyq6p6ZBL7ntta+N/f3GfsajJrxQmnH7Xv/JP87yf1aa9fuxnyz2ubx\nXJfJgrvj/7o/I8n3k7z7dBeB2ys7vH/ukclr5Y9srX16N+abxazHUjNclG8I89w3Y1uDMf3j4dtJ\nnnX0WoWqeluSfa21Zww1204tyxqMqrosyVOSnNda+/LQ8/RUVX+Y5IuttZcMPcssquppSf5rJn/k\nHfnj9YxMXsK6JZN1cacND6N/iWR68ZWZLsBylAcnuTWThUULYbvHU1UXJ/lXR226e5KDSZ6d5Ord\nmW52O7x/jqyPuW2ncXZklmOpYy/K96LdnGteO7xvRqG19r2q2kzyuCQfSP7mdPzjkizNYu+xmoaL\npyV5zLKFi6nbZEF+fs3oI0l+8rhtb0tyKMlrthMukiUIGNtVVY/IpA71sUxeG3tUkkuSvLO1dnjI\n2ebRWvvzoz+vqpszSZpfaq19dZip5ldVD0vy0CSfSHJTkvskeXWSazOyRVLTMxdXZnKm6VeTnDP5\nnTbei/JV1Y8kuVOSeyU5o6oeOP3SF1trNw832bZckuRt06BxdSa14dtl8gNzVKrq9pk8N478VXnv\n6X3xzdbaV06+5+KpqsuTrCd5apKbp4vWk+Rwa210V76uqt/M5OXQLye5QyaL1B+T5IlDzjWP6XP6\nmLUw098x32itHdru91mZgJHkrzN5fe9VmSTK6zLpxl865FCdjfn1rm9n8t4Xv55J1/+GTJ6sv7GT\n1wAHsowX5Xt1kucd9fnW9L+PTfLxvR9n+1pr751WIF+dyXvgXJPk/Nba14adbC4PyeSPpDb9eP10\n+9uzoGfKTuGiTI7hyuO2vzDJO/Z8mp07J5P74W5JDif5XJInztvAWEAz/34Z/RoMAGDxjL5FAgAs\nHgEDAOhOwAAAuhMwAIDuBAwAoDsBAwDoTsAAALoTMACA7gQMAKA7AQMA6E7AAAC6EzCAPVdVd6mq\nG6rqFUdte1RV/XVVPXbI2YA+XOwMGERV/UyS/5bkkUn+NJOrnL6/tfYrgw4GdCFgAIOpqjdlcnn7\n/5XkAUke2lr73rBTAT0IGMBgqursJH+S5J5J9rfWPj/wSEAn1mAAQ7pPkrtn8rPoxwaeBejIGQxg\nEFV1VpKrk3wmyf9JciDJA1prXx90MKALAQMYRFX9dpJnJvmpJN9OcmWSb7XWnjLkXEAfXiIB9lxV\nPSbJLyW5sLV2c5v8pfO8JP+wql487HRAD85gAADdOYMBAHQnYAAA3QkYAEB3AgYA0J2AAQB0J2AA\nAN0JGABAdwIGANCdgAEAdCdgAADdCRgAQHcCBgDQ3f8H4W3BsmGaQbEAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x192ced6c470>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "def relu_transfer(t):\n",
    "    return np.maximum(0,x)\n",
    "\n",
    "x = np.arange(-5.0, 5.0, 0.02)\n",
    "\n",
    "plt.ylim([-2,3])\n",
    "plt.xlim([-5,4])\n",
    "plt.xlabel(\"x\")\n",
    "plt.ylabel(\"y\")\n",
    "plt.plot(x, relu_transfer(x), 'r')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[10 10 10 10 10]\n"
     ]
    }
   ],
   "source": [
    "x = np.array([1,2,3,4,5])\n",
    "\n",
    "print(np.maximum(x,10))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Sigmoid Transfer Function\n",
    "\n",
    "** Equation 1.4: The Sigmoid Transfer Function **\n",
    "$$ f(x) = \\frac{1}{1 + e^{-x}} $$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAhkAAAF5CAYAAAAyBjhMAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAAPYQAAD2EBqD+naQAAIABJREFUeJzt3Xd4nmX5//H3SS3bb8uyLBkFZSmj/YJU4YuAgGwUASOg\nUPYSi8hPEIEWcCEgFSuIMsoIsjeUoeyW0TJkVJBSypBSBMoqdHD9/rhSG0rSJiVPrme8X8fxHEnu\n3vfTT56myZnzvkaklJAkSepq85UOIEmS6pNFhiRJqgiLDEmSVBEWGZIkqSIsMiRJUkVYZEiSpIqw\nyJAkSRVhkSFJkirCIkOSJFWERYYkSaqIqigyImLjiLguIl6OiI8iYocOXPP1iBgdER9ExDMR8YPu\nyCpJkjqmKooMYBHgUeBgYK6bqUTESsANwB3AOsAZwJ8jYovKRZQkSZ0R1bZBWkR8BOyUUrpuDuf8\nGtg6pbR2q2PNQK+U0jbdEFOSJM1FtXQyOmtD4PbZjo0ABhTIIkmS2lCrRcbSwMTZjk0E/iciFiiQ\nR5IkzeYzpQN0l4hYAtgKGA98UDaNJEk1ZUFgJWBESuk/Hb2oVouMV4E+sx3rA7ydUvqwnWu2Ai6u\naCpJkurb7sAlHT25VouMkcDWsx3bsuV4e8YDXHTRRayxxhoVilV/Bg0axOmnn146Rs3xdes8X7N5\n4+vWeRV7zT78EN56Kz/efHPW+5Mnw7vvwjvvzHrb+vHuu5ASbLEF/OpXXZ+rCzz99NPsscce0PKz\ntKOqosiIiEWAVYFoOdQ3ItYB3kgpvRgRvwSWTSnNXAvjLOCQllkm5wKbA98B5jSz5AOANdZYg379\n+lXi06hLvXr18vWaB75unedrNm983TqvU6/ZtGnw6qvwyiv58fLLs95/7TV4/XWYNCm/fffdT14/\n//ywxBLQu/esx9JLz3q/V69Z76+yClT/v2WnhhtURZEB/C/wd/IaGQk4teX4BcBA8kDPz888OaU0\nPiK2BU4Hfgi8BOyTUpp9xokkSW1LCd54A8aPh+ef//jb1oVEaz17wrLLwjLL5GJhnXVgqaVgySXz\nY/b3F1kEItr4yxtDVRQZKaW7mMNMl5TS3m0cuxvoX8lckqQal1LuNIwdO+vx4IOw9tq5mHjnnVnn\nLroorLwyrLQSDBiQi4nZH0ss0dBFQ2dVRZEhSdKnklK+rfHoo/CPf3y8qHjzzXzOfPPlWxIpwcYb\nw5575qJiZmGx+OIWEF3MIkNz1NTUVDpCTfJ16zxfs3nTkK/bjBnwz3/mgqL1Y9Kk/OeLLgprrAGr\nrw7bbZffrr56LjAWWICm5mZoxNetgKpbVrxSIqIfMHr06NEOkpKkWvLaa/DAAzBqVH48+OCsQZYr\nrgjrrpsf66yT3660kh2JLjZmzBj69+8P0D+lNKaj19nJkCRVlxdegL//PT/uvRfGjcvH+/TJYyWO\nPRY22CAXFIstVjar5sgiQ5JU1quvwh13zCosxo3LnYh11oHtt8+FxYYbwgor2KGoMRYZkqTu9dFH\n8NBDcOON+TGmpfu+1lqw7baw6aawySZ5IKZqmkWGJKnyPvwQbr0VrrwSbr45j7Po3Ru++U0YNCiv\ndtln9t0iVOssMiRJlTGzsLjsMrjuOnj77TzrY++9c8diwAD4jD+G6pn/upKkrpMS3H03nH8+XHVV\nLizWXBOOOAJ22SW/r4ZhkSFJ+vReeAGGD8/FxbhxeU2KQYNg110tLBqYRYYkad5Mnw7XXw/DhuXZ\nIQsvnLsV552XV9R0JkjDs8iQJHXOf/4Df/5zLi4mTMhjK/7yl1xgLLpo6XSqIhYZkqSOefZZOOUU\nuPDCPPaiqQkOPRT6u1el2maRIUmas0cegV/+Eq64Ik8z/fnPYb/98lbm0hxYZEiS2jZqFAweDLfc\nAn37wh//CD/4ASy4YOlkqhHzlQ4gSaoyjz8OO+yQx1q89BJcckne9fSAAyww1CkWGZKk7Nln4Xvf\nyxuPPf10Li4eeyyPvXDRLM0DiwxJanRvvZUXy1pzzbyQ1tlnw1NP5eJiPn9MaN5ZmkpSo5oxA849\nF372M3j/fRgyJC+g5S0RdRFLVElqRPffD+uvD/vvnzcpe+YZOPpoCwx1KYsMSWokb78NhxwCX/sa\n9OgBI0fm5cCXXbZ0MtUhb5dIUqO4/no46KA8BmPoUDj44FxoSBViJ0OS6t2kSbDbbnla6tprw5NP\nwmGHWWCo4uxkSFI9u/lm2HvvvJnZxRfnGSNuXKZuYidDkurRlCm5W7HNNrDeevDEE3kNDAsMdSM7\nGZJUbx59FHbfHZ57Lo+9OPRQiwsVYSdDkupFSvCnP8FXvpJX6Bw9OnczLDBUiEWGJNWDKVNg4MC8\nv8jAgfDgg7DWWqVTqcF5u0SSat24cbDzzjB2LFxwAXz/+6UTSYBFhiTVtltuyTNGllgib82+zjql\nE0n/5e0SSapFKeVBndtum1fvfPhhCwxVHYsMSao106fnGSOHH553T732Wujdu3Qq6RO8XSJJtWTy\nZNh1V/jb3+Ccc2DffUsnktplkSFJteLf/847pk6YkMdibL556UTSHFlkSFItePZZ2HJLmDYN7r3X\n6amqCY7JkKRqN2ZMHty5wAJw//0WGKoZFhmSVM3uugu+/nVYaaXcwVhhhdKJpA6zyJCkanXHHbD1\n1rDBBnmg55JLlk4kdYpFhiRVoxEjYLvtYJNN4PrrYdFFSyeSOs0iQ5KqzU03wQ475NkjV18NCy1U\nOpE0TywyJKma3HQT7LRTvk1y1VWw4IKlE0nzzCmsklQt7rwzb3S29dZw+eXQs2fpRNKnYidDkqrB\nAw/A9tvDRhvBX/9qgaG6YJEhSaU99lheyXOddeCaa7xForphkSFJJf3rX7DFFrDKKnDjjbDIIqUT\nSV3GIkOSSpk4EbbaChZfPO9F0qtX6URSl3LgpySV8O67eR2M99+HkSNdaEt1ySJDkrrbtGl5u/ax\nY+Huu/OS4VIdssiQpO6UEhx0ENx2Wx6Dsd56pRNJFWORIUnd6ZRT4C9/gQsuyFu3S3XMgZ+S1F2u\nuw5++lM45hj4/vdLp5EqrmqKjIg4JCKej4gpETEqItafy/k/ioixEfF+REyIiNMiYoHuyitJnfKP\nf8Duu+clw088sXQaqVtURZEREbsBpwLHA+sBjwEjIqLN4dYR8T3gly3nrw4MBHYDTu6WwJLUGa+9\nllfzXHVVGD4c5quKb71SxVXLV/og4OyU0vCU0ljgQOB9cvHQlgHAvSmlv6aUJqSUbgeagQ26J64k\nddDUqXk/kg8+yLdL3LJdDaR4kRERPYH+wB0zj6WUEnA7uZhoy/1A/5m3VCKiL7ANcGNl00pSJ/3k\nJ3lfkquvhs9/vnQaqVtVw+ySJYEewMTZjk8EVmvrgpRSc8utlHsjIlquPyul9OuKJpWkzmhuhqFD\n4cwzYUB7vzNJ9at4J2NeRMTXgWPIt1XWA74NbBcRx5bMJUn/9cQTsO++ebDnwQeXTiMVUQ2djNeB\nGUCf2Y73AV5t55ohwPCU0nktHz8ZEYsCZwMnzekvGzRoEL1m2x+gqamJpqamzuaWpLa9/XYeh9G3\nL5x9NkSUTiR1WHNzM83NzR87Nnny5Hl6ruJFRkppWkSMBjYHrgNouQWyOTC0ncsWBj6a7dhHM69t\nGdPRptNPP51+/fp96tyS1KaUYO+94dVX4eGH3VVVNaetX7zHjBlD//79O/1cxYuMFqcB57cUGw+S\nZ5ssDJwPEBHDgZdSSse0nH89MCgiHgUeAL5A7m5cN6cCQ5Iq7rTT4Kqr4Jpr4AtfKJ1GKqoqioyU\n0mUtAzmHkG+TPApslVKa1HLK8sD0VpecSO5cnAgsB0wid0EckyGpnIceyit6Hnkk7Lhj6TRScVVR\nZACklIYBw9r5s81m+3hmgeGyeZKqwzvvQFMTrLsunOy6gBJUUZEhSTXtkENg4kS45RaYf/7SaaSq\nYJEhSZ/WRRfBhRfmx6qrlk4jVY2aXCdDkqrGc8/BQQfBnnvCHnuUTiNVFYsMSZpX06blcRhLLw1/\n+EPpNFLV8XaJJM2rk06CRx6BkSPhs58tnUaqOnYyJGlePPRQnkXy85/D//5v6TRSVbLIkKTOmjIF\nfvADWG89OPro0mmkquXtEknqrGOPhXHjYMwY6NmzdBqpallkSFJn3H03nH46nHIKrLlm6TRSVfN2\niSR11DvvwF57wUYbwY9+VDqNVPXsZEhSR/3kJ/Daa3DbbdCjR+k0UtWzyJCkjrjzTjj77Lwexiqr\nlE4j1QRvl0jS3EyZAvvtl2+THHhg6TRSzbCTIUlzM2QITJgA118P8/m7mdRR/m+RpDl59NE8k+Tn\nP4fVVy+dRqopFhmS1J7p02GfffJU1aOOKp1GqjneLpGk9vzud3lvklGjYP75S6eRao6dDElqy3PP\nwXHHweGHwwYblE4j1SSLDEmaXUpwyCHwuc/BiSeWTiPVLG+XSNLsrrwSRoyA666DRRctnUaqWXYy\nJKm1d97JS4bvsANsv33pNFJNs8iQpNaGDIE33oAzziidRKp5FhmSNNMTT+QdVo89FlZaqXQaqeZZ\nZEgS5MGeBx8Mq64KP/5x6TRSXXDgpyQBXHgh3HMP3H47LLBA6TRSXbCTIUlvvQVHHgnf/S5svnnp\nNFLdsMiQpMGD4f334dRTSyeR6opFhqTG9vTTcOaZebDnssuWTiPVFYsMSY0rJRg0CFZYIa+NIalL\nOfBTUuO66aa8sufVV8OCC5ZOI9UdOxmSGtPUqbmLsfnmsOOOpdNIdclOhqTG9Pvf551Wr7oKIkqn\nkeqSnQxJjWfixLx8+EEHwZe+VDqNVLcsMiQ1nmOPhR498tRVSRXj7RJJjWXMGPjLX/IGaEssUTqN\nVNfsZEhqHCnBEUfA6qvDgQeWTiPVPTsZkhrHDTfAXXfBjTdCz56l00h1z06GpMYwfTocdRRsthls\nvXXpNFJDsJMhqTH85S8wdixcfLFTVqVuYidDUv175x04/njYYw/o1690GqlhWGRIqn+//W3ezv2k\nk0onkRqKRYak+vbKK7nIOPxwWHHF0mmkhmKRIam+HX88LLQQHH106SRSw3Hgp6T69cQTcO65cNpp\n0Lt36TRSw7GTIal+HXUUrLxy3qNEUrezkyGpPt11F9x8M1x2Gcw/f+k0UkOykyGp/qSUx2D07w/f\n+U7pNFLDspMhqf7ccAOMHAkjRrjwllSQnQxJ9WXGDDjmGNh0U9hii9JppIZmJ0NSfWluzrNKRo60\niyEVZidDUv2YOhWOOw523BE23LB0GqnhVU2RERGHRMTzETElIkZFxPpzOb9XRPwhIl6JiA8iYmxE\nfLO78kqqQuecA+PHw8knl04iiSq5XRIRuwGnAvsDDwKDgBER8cWU0uttnN8TuB14Ffg28AqwIvBW\nt4WWVF3eew9OPBH23BPWWqt0GklUSZFBLirOTikNB4iIA4FtgYHAb9o4fx+gN7BhSmlGy7EJ3RFU\nUpUaOhTeeAMGDy6dRFKL4rdLWroS/YE7Zh5LKSVyp2JAO5dtD4wEhkXEqxHxj4g4OiKKfz6SCnjz\nTfjNb+CAA2CllUqnkdSiGjoZSwI9gImzHZ8IrNbONX2BzYCLgK2BVYE/kj+fEysTU1LV+vWvYdo0\nOPbY0kkktVINRca8mI9chOzf0vV4JCKWB45kLkXGoEGD6NWr18eONTU10dTUVKmskirp3//Ot0qO\nOAL69CmdRqp5zc3NNDc3f+zY5MmT5+m5Iv+MLqfldsn7wM4ppetaHT8f6JVS+lYb19wJTE0pbdnq\n2DeBG4EFUkrT27imHzB69OjR9OvXr8s/D0mFHHwwXHopjBvnTqtShYwZM4b+/fsD9E8pjenodcXH\nMKSUpgGjgc1nHouIaPn4/nYuu498i6S11YB/t1VgSKpT48fnaas//akFhlSFihcZLU4D9ouI70fE\n6sBZwMLA+QARMTwiftHq/D8Ci0fE0Ij4QkRsCxwNnNnNuSWVdNJJsPjicMghpZNIakNVjMlIKV0W\nEUsCQ4A+wKPAVimlSS2nLA9Mb3X+SxGxFXA68Bjwcsv7bU13lVSPnnsOzj8/zypZZJHSaSS1oSqK\nDICU0jBgWDt/tlkbxx4AvlrpXJKq1EknwVJLwYEHlk4iqR1VU2RIUof9619w4YVw6qmw8MKl00hq\nR7WMyZCkjjvxRPjc52D//UsnkTQHdjIk1ZZnnoGLLoLf/Q4WWqh0GklzYCdDUm058URYZhnYb7/S\nSSTNhZ0MSbVj7Fi45BL4/e9hwQVLp5E0F3YyJNWOIUNgueVgn31KJ5HUAXYyJNWGp57Ky4cPGwYL\nLFA6jaQOsJMhqTYMGQKf/zwMHFg6iaQOspMhqfo98QRcdhmcfTbMP3/pNJI6yE6GpOo3eDCsuCLs\ntVfpJJI6wU6GpOr2+ONwxRXw5z9Dz56l00jqBDsZkqrb4MHQty98//ulk0jqJDsZkqrXo4/CVVfB\nuefaxZBqkJ0MSdVr8GBYZRXYc8/SSSTNAzsZkqrTmDFwzTVwwQXwGb9VSbXIToak6nTCCfCFL8D3\nvlc6iaR55K8HkqrPww/D9dfDhRfaxZBqmJ0MSdXnhBNgtdWgqal0Ekmfgr8iSKouDzwAN96Yd1vt\n0aN0Gkmfgp0MSdXlhBNgjTVg111LJ5H0KdnJkFQ9Ro6EW27Ju63axZBqnp0MSdXjhBNgrbVgl11K\nJ5HUBexkSKoO990Ht94Kl18O8/n7j1QPOv0/OSIuiIj/q0QYSQ3s+ONh7bXh298unURSF5mXTkYv\n4PaIeAE4D7ggpfRy18aS1FDuuQfuuAOuvNIuhlRHOv2/OaW0E7Ac8EdgN2B8RNwcEd+JCHcwktR5\nxx8P66wDO+1UOomkLjRPvzKklCallE5LKa0DfAX4F3Ah8EpEnB4RX+jKkJLq2F13wd//njdDs4sh\n1ZVP9T86IpYBtmh5zABuAr4MPBURgz59PEl1LSU47jjo1w922KF0GkldrNNjMlpuiewA7A1sCTwO\n/A64JKX0dss5TcAw4PSuiyqp7vz973D33XDddRBROo2kLjYvAz//Te6ANAMbpJQebeOcW4C3Pk0w\nSXUupTwW43//F7bbrnQaSRUwL0XGIODylNIH7Z2QUnoTWHmeU0mqf3fcAffeCzfcYBdDqlOdLjJS\nShdWIoikBjKzi7HBBrDNNqXTSKoQV/yU1P1uvRXuvx9uvtkuhlTHnC8mqXvN7GJsuCFstVXpNJIq\nyE6GpO51yy3wwAMwYoRdDKnO2cmQ1H1mdjG++lXYYovSaSRVmJ0MSd3nxhvhoYfyzBK7GFLds5Mh\nqXvMXN1zk01g001Lp5HUDexkSOoe114LjzwCd95pF0NqEHYyJFXeRx/lsRibbZY7GZIagp0MSZV3\n1VXw+ONwzz2lk0jqRnYyJFXWRx/BCSfAllvCRhuVTiOpG9nJkFRZl18OTz4Jf/5z6SSSupmdDEmV\nM2NG7mJss01e4VNSQ7GTIalyLr0Uxo6F4cNLJ5FUgJ0MSZUxfToMHgzbbw/rr186jaQC7GRIqoyL\nL4Znn4W//rV0EkmF2MmQ1PWmTYMhQ+Bb34L11iudRlIhdjIkdb0LL4Rx4+Dqq0snkVSQnQxJXWvq\nVDjxRNhlF1h77dJpJBVkJ0NS1zr/fHjhBbjhhtJJJBVWNZ2MiDgkIp6PiCkRMSoiOjQcPSK+GxEf\nRcRVlc4oaS4++ABOOgl23RXWWqt0GkmFVUWRERG7AacCxwPrAY8BIyJiyblctxJwCnB3hSNK6oiz\nz4ZXXsmDPiU1vKooMoBBwNkppeEppbHAgcD7wMD2LoiI+YCLgOOA57slpaT2vfsunHwy7LUXfPGL\npdNIqgLFi4yI6An0B+6YeSyllIDbgQFzuPR4YGJK6bzKJpTUIWecAZMnw3HHlU4iqUpUw8DPJYEe\nwMTZjk8EVmvrgojYCNgbWKey0SR1yBtvwCmnwEEHwQorlE4jqUoU72R0VkQsCgwH9kspvVk6jyRy\ngTF9Ohx9dOkkkqpINXQyXgdmAH1mO94HeLWN81cBVgSuj4hoOTYfQERMBVZLKbU7RmPQoEH06tXr\nY8eamppoamqat/RSo3v11Xyr5IgjoM/s/40l1Zrm5maam5s/dmzy5Mnz9FyRhz+UFRGjgAdSSoe3\nfBzABGBoSumU2c6dH1h1tqc4GVgU+CHwbEppeht/Rz9g9OjRo+nXr18FPgupQR12GFx0ETz/PPTu\nXTqNpAoYM2YM/fv3B+ifUhrT0euqoZMBcBpwfkSMBh4kzzZZGDgfICKGAy+llI5JKU0Fnmp9cUS8\nRR4v+nS3ppYa3fjxedrqkCEWGJI+oSqKjJTSZS1rYgwh3yZ5FNgqpTSp5ZTlgU90JyQVNngwLL54\n7mZI0myqosgASCkNA4a182ebzeXavSsSSlL7nn4ahg+HoUNhkUVKp5FUhWpudomkKnHccXm66n77\nlU4iqUpVTSdDUg0ZPRquuCJvhjb//KXTSKpSdjIkdd7PfgZrrAF77FE6iaQqZidDUufccQeMGAFX\nXgk9epROI6mK2cmQ1HEffQRHHQUDBsC3vlU6jaQqZydDUsddeimMGQP33gv/XXBXktpmJ0NSx3z4\nYR6LsdNO8LWvlU4jqQbYyZDUMcOGwYsvws03l04iqUbYyZA0d2+9BSedBPvuC6uvXjqNpBphkSFp\n7n71K/jgAzj++NJJJNUQiwxJc/bii3kr9yOPhGWWKZ1GUg2xyJA0Z8cdB//zP7nIkKROcOCnpPY9\n/jhccAGceSZ89rOl00iqMXYyJLUtJTjiCPjCF9wETdI8sZMhqW3XX5+XEL/hBujZs3QaSTXIToak\nT5o6FX78Y9hyS9hmm9JpJNUoOxmSPunMM2HcOLj6apcPlzTP7GRI+rhJk2DIEDjgAPjSl0qnkVTD\nLDIkfdzMBbcGDy6bQ1LN83aJpFmeeALOPhtOOQWWWqp0Gkk1zk6GpGzmlNW+feHQQ0unkVQH7GRI\nym66CW67Da65Buafv3QaSXXAToYk+PBDGDQINtsMdtihdBpJdcJOhiQ49VR4/nmnrErqUnYypEY3\nYQKcdBL88Iew1lql00iqIxYZUqM74gjo3XvW1FVJ6iLeLpEa2a23wpVXwsUX5+3cJakL2cmQGtWH\nH8Jhh8Emm0BTU+k0kuqQnQypUZ1+Ojz3XO5kONhTUgXYyZAa0Ysvwokn5sGe7k8iqUIsMqRGdNhh\n0KsXnHBC6SSS6pi3S6RGc/XVcO21cPnlDvaUVFF2MqRG8vbbeV+S7baDnXcunUZSnbPIkBrJMcfA\n5Mnwhz842FNSxXm7RGoUo0bBsGFw2mmwwgql00hqAHYypEYwbRrsvz/065cHfUpSN7CTITWCU0+F\np56Chx6CHj1Kp5HUIOxkSPXumWdg8GD40Y9gvfVKp5HUQCwypHo2YwbsvTcsvzwMGVI6jaQG4+0S\nqZ6dcQaMHAl33QULL1w6jaQGYydDqlf//Cf87Gd56fCNNy6dRlIDssiQ6tGMGTBwYL5N8otflE4j\nqUF5u0SqRzNvk9x9t7dJJBVjJ0OqNzNvkxx+OGy0Uek0khqYRYZUT6ZNgx/8IN8mOfnk0mkkNThv\nl0j15KST4OGH4b77vE0iqTg7GVK9uO++XGQcfzx85Sul00iSRYZUFyZPhj32gAED4OijS6eRJMDb\nJVJ9OPRQ+M9/4G9/g8/431pSdfC7kVTrLrkELroILrwQVl65dBpJ+i9vl0i1bPx4OOggaGqC3Xcv\nnUaSPqZqioyIOCQino+IKRExKiLWn8O5+0bE3RHxRsvjtjmdL9WlDz+EXXeFxRaDYcMgonQiSfqY\nqigyImI34FTgeGA94DFgREQs2c4lmwCXAF8HNgReBG6NiGUqn1aqEj/5CTz2GFx+OfTuXTqNJH1C\nVRQZwCDg7JTS8JTSWOBA4H1gYFsnp5T2TCmdlVJ6PKX0DLAv+XPZvNsSSyVdfjn8/vdw2mmwvk08\nSdWpeJERET2B/sAdM4+llBJwOzCgg0+zCNATeKPLA0rV5plnYJ99YLfd4OCDS6eRpHYVLzKAJYEe\nwMTZjk8Elu7gc/waeJlcmEj1a8oU+M53YJll4JxzHIchqarV/BTWiPgpsCuwSUppauk8UsWklNfD\n+Ne/4IEH4LOfLZ1IkuaoGoqM14EZQJ/ZjvcBXp3ThRFxJHAUsHlK6cmO/GWDBg2iV69eHzvW1NRE\nU1NThwNLRZx1Fpx7Lpx3Hnz5y6XTSKpTzc3NNDc3f+zY5MmT5+m5Ig9/KCsiRgEPpJQOb/k4gAnA\n0JTSKe1ccxRwNLBlSumhDvwd/YDRo0ePpl+/fl0XXuoOd90F3/hGHoNxxhml00hqMGPGjKF///4A\n/VNKYzp6XTV0MgBOA86PiNHAg+TZJgsD5wNExHDgpZTSMS0f/z9gMNAETIiImV2Qd1NK73Vzdqmy\nXnghj8PYeGP47W9Lp5GkDquKIiOldFnLmhhDyLdJHgW2SilNajlleWB6q0sOJM8muWK2pxrc8hxS\nfXjvPdhpJ1h0UbjsMujZs3QiSeqwqigyAFJKw4Bh7fzZZrN97AYNqn8ffQQDB+YpqyNHwpLtrU0n\nSdWpaooMSbM59ti86NYVV8Daa5dOI0mdZpEhVaNzzoFf/jKPwfj2t0unkaR5Ug2LcUlqbcSIvLPq\nQQfBEUeUTiNJ88wiQ6omjz8Ou+wCW20FQ4e6oqekmmaRIVWL55+Hb34TVlkFLr0UPuPdTEm1zSJD\nqgYTJ8KWW8JCC8FNN7lkuKS64K9KUmmTJ+cOxnvvwb335s3PJKkOWGRIJU2ZAttvD+PHw913Q9++\npRNJUpexyJBK+eAD2HlnGD0abr/dTc8k1R2LDKmEDz/MBcbf/w7XXw8DBpROJEldziJD6m4zC4w7\n7sgFxje+UTqRJFWEs0uk7jR1al4H4/bb4dprYYstSieSpIqxkyF1lylT8pbtMwuMrbYqnUiSKsoi\nQ+oOb7+dZ5E8/HC+RbLllqUTSVLFWWRIlfb667D11vDss3DbbfDVr5ZOJEndwiJDqqSXX85di9df\nhzvvhHXR213GAAAMk0lEQVTXLZ1IkrqNAz+lSnn8cfjKV+Ddd+GeeywwJDUciwypEm67DTbaCPr0\ngVGj4ItfLJ1IkrqdRYbU1c47D7bZBjbeGO66y71IJDUsiwypq8yYAUcdBQMHwj775Gmqiy5aOpUk\nFePAT6krvPkmfPe7eRXP00+Hww+HiNKpJKkoiwzp03rySdhxx1xojBgBm29eOpEkVQVvl0ifxiWX\nwIYbwsILw0MPWWBIUisWGdK8eP992Hdf2H333MW4/37o27d0KkmqKt4ukTrrySdh111h/Hg491zY\nay/HX0hSG+xkSB2VEvzpT7D++jDffPn2yN57W2BIUjssMqSOeOmlvP/IAQfAnnvCAw/AmmuWTiVJ\nVc3bJdKcpATDh+cpqYssAjfdlIsNSdJc2cmQ2vPyy3lQ51575bdPPGGBIUmdYJEhzW769Lyg1uqr\nw4MPwjXXwAUXwGKLlU4mSTXFIkNq7f77oX9/+PGP4Qc/gLFjcxdDktRpFhkSwKuv5v1GvvY1WGCB\nPHPkzDOhd+/SySSpZjnwU43t3Xfh1FPhlFNycTFsGOy/P/ToUTqZJNU8iww1punT85bsxx2X9xz5\n4Q/hmGPsXEhSF/J2iRrLjBlw8cXw5S/njsXmm8M//wm/+Y0FhiR1MYsMNYbp0+Gii/ICWnvsAaus\nAg8/nI+tuGLpdJJUlywyVN8++ADOOScXF3vuCautlgd13nBDnkUiSaoYx2SoPr32Gvzxj/CHP8Dr\nr8NOO0Fzs4WFJHUjiwzVl9Gj4ayz4MIL8wyRvfeGH/0IVl21dDJJajgWGap977wDl1ySd0gdMwaW\nWy7PGjnwQFh88dLpJKlhWWSoNn30EYwcmZf7vuQSmDIFtt0WBg+Gb34TPuOXtiSV5ndi1ZZ//CMX\nFc3N8MIL8PnPw09+klfrXH750ukkSa1YZKi6pQRPPZU3Kbv00rwT6uKLwy67wPe+BxttBPM5SUqS\nqpFFhqrPjBlw331w7bX58dxzsMgisN128ItfwFZbwfzzl04pSZoLiwxVhxdfhNtuy4/bb8/TTvv0\nyTugDh0Km20GCy5YOqUkqRMsMlTGW2/BPffMKizGjoWIvI7FfvvBDjvABht4K0SSaphFhiovpTxI\n87774N5789snnsjHV14ZttgCTjwRNt0UlliidFpJUhexyFDXmzQpr1cxenR+O2oUvPxy/rM11oCv\nfQ1+/OM8aHOVVcpmlSRVjEWG5t2MGTB+PDz5JDz22KzC4sUX85/36gX9+sHuu+eC4qtftVMhSQ3E\nIkNzN3UqPP98nkra+jF2bN6ADGCxxXJB0dSUx1X06wd9+zqmQpIamEWGsrfeylNFx4375NsJE/IK\nmwC9e8Naa8H668Nee+XdTddcE5ZdNg/clCSpRdUUGRFxCHAksDTwGHBYSumhOZy/CzAEWAl4Bvhp\nSunmbohaW1LKBcRLL+XHyy+3/f5bb826plevPFaib1+aF1mEpqOPzhuMrblmnlZqMTFXzc3NNDU1\nlY5RU3zN5o2vW+f5mnWfqigyImI34FRgf+BBYBAwIiK+mFJ6vY3zvwpcAvw/4EZgd+CaiFgvpfRU\n9yUvICV4//28jsRrr+VBlpMmzXp/9mOvvZb39ZgpApZeOi/Bvfzy8PWv5w3FVl45395YZZV866Ol\nkGjeYQea9t+/zOdaw/wm1nm+ZvPG163zfM26T1UUGeSi4uyU0nCAiDgQ2BYYCPymjfN/CNycUjqt\n5ePjImIL4FDg4G7I23UmTMgDJ996Kz/efHPW++09pk375PP06gVLLQWf+1x+u+66sz5ebrlcUCy3\nXC4wevbs/s9TktRwihcZEdET6A/8YuaxlFKKiNuBAe1cNoDc+WhtBLBjRUJW0hVX5OmckHcOXWyx\nPO6h9WPFFT95bKmlZhURSy4JCyxQ9vOQJGk2xYsMYEmgBzBxtuMTgdXauWbpds5fumujdYOBA+G7\n382Fw0ILOd5BklQ3qqHI6C4LAjz99NOlc7Tt1VdLJ2jT5MmTGTNmTOkYNcfXrfN8zeaNr1vn+Zp1\nXqufnZ3aRCpSSl2fpjMB8u2S94GdU0rXtTp+PtArpfStNq55ATg1pTS01bETgB1TSuu18/d8D7i4\na9NLktRQdk8pXdLRk4t3MlJK0yJiNLA5cB1ARETLx0PbuWxkG3++Rcvx9owgz0IZD3zw6VJLktRQ\nFiQvGTGiMxcV72QARMSuwPnAgcyawvodYPWU0qSIGA68lFI6puX8AcCdwNHkKaxNwE+BfnU/hVWS\npBpRvJMBkFK6LCKWJC+u1Qd4FNgqpTSp5ZTlgemtzh/Zcvvj5JbHs+RbJRYYkiRViaroZEiSpPrj\n7lWSJKkiLDIkSVJFNGyRERHbRsSoiHg/It6IiKtKZ6oFETF/RDwaER9FxNql81SziFgxIv4cEeNa\nvs6ejYgTWqZtq5WIOCQino+IKS3/L9cvnalaRcTREfFgRLwdERMj4uqI+GLpXLUkIn7a8j3stLmf\n3dgiYtmIuDAiXm/5PvZYRPTr6PUNWWRExM7AcOAvwJeBmRuuae5+A7wEOJhn7lYHAtgPWJM8a+pA\n8mBltWi1QeLxwHrkXZhHtAwG1ydtDPwe+ArwDaAncGtELFQ0VY1oKWD3J3+daQ4iojdwH/AhsBWw\nBvBj4M0OP0ejDfyMiB7ktTJ+nlI6v2ya2hIRWwO/BXYGngLWTSk9XjZVbYmII4EDU0qrls5SLSJi\nFPBASunwlo8DeBEYmlJqa4NEtdJSjL0G/F9K6d7SeapZRCwKjAYOAn4OPJJSOqJsquoVEb8CBqSU\nNpnX52jETkY/YFmAiBgTEa9ExE0RsVbhXFUtIvoAfwL2AKbM5XS1rzfwRukQ1aLVBol3zDyW8m8+\nc9ogUR/Xm9xZ9Otq7v4AXJ9S+lvpIDVie+DhiLis5dbcmIjYtzNP0IhFRl9yC/t48roc25JbP3e2\ntIbUtvOAYSmlR0oHqVURsSpwKHBW6SxVZE4bJNbehofdrKXr8zvgXtcJmrOI+C6wLnkRR3VMX3LX\n55/AlsAfgaERsWdHn6BuioyI+GXLQJ72HjNaBkfN/JxPSild0/JDc2/ybwK7FPsECujoaxYRPwQW\nBX4989KCsYvrxNda62uWA24G/ppSOrdMctWhYeTxPt8tHaSaRcTy5GJs95TStNJ5ash8wOiU0s9T\nSo+llM4BziGPLeuQqljxs4v8lvzb9pyMo+VWCfDfLeVSSlMjYhywQoWyVauOvGbPA5uSW9cfxse3\non84Ii5OKe1doXzVqqNfa0AenQ38jfzb5gGVDFaDXgdmkFf6ba0PUJ1bE1eJiDgT2AbYOKX079J5\nqlx/YClgTMz6JtYD+L+IOBRYIDXaAMWO+Tetfla2eBr4dkefoG6KjJTSf4D/zO28ls3YPgRWA+5v\nOdaTvPHLCxWMWHU68ZodBvys1aFlyZvk7Erea6ahdPR1g/92MP4GPAQMrGSuWjSPGyQ2vJYCY0dg\nk5TShNJ5asDt5JmErZ1P/oH5KwuMdt1H/lnZ2mp04mdl3RQZHZVSeicizgIGR8RL5BfrKPLtksuL\nhqtSKaWXWn8cEe+Rb5mMSym9UiZV9WvpYNxJ7gYdBXxu5i9RKaXZxyA0stOA81uKjZkbJC5M/iGg\n2UTEMPKmkDsA77UMygaYnFJyh+k2pJTeI8+I+6+W72P/SSnN/pu6ZjkduC8ijgYuI0+b3pc8Lb9D\nGq7IaHEkMI28VsZCwAPAZimlyUVT1RYr/7nbgjxwqi95Sibk4iyRW7WiQxsk6uMOJH8N3Tnb8b3J\n39PUMX4Pm4uU0sMR8S3gV+Qpv88Dh6eULu3oczTcOhmSJKl71M3sEkmSVF0sMiRJUkVYZEiSpIqw\nyJAkSRVhkSFJkirCIkOSJFWERYYkSaoIiwxJklQRFhmSJKkiLDIkSVJFWGRIkqSKsMiQVERELBkR\n/46In7Y69tWI+DAiNi2ZTVLXcIM0ScVExNbANcAA4BnyDqxXp5R+UjSYpC5hkSGpqIj4PbAF8DDw\nJWD9lNK0sqkkdQWLDElFRcSCwBPA8kC/lNJThSNJ6iKOyZBU2qrAsuTvRysXziKpC9nJkFRMRPQE\nHgQeAf4JDAK+lFJ6vWgwSV3CIkNSMRFxCvBtYG3gfeBO4O2U0vYlc0nqGt4ukVRERGwC/BDYI6X0\nXsq/8Xwf2CgiDiibTlJXsJMhSZIqwk6GJEmqCIsMSZJUERYZkiSpIiwyJElSRVhkSJKkirDIkCRJ\nFWGRIUmSKsIiQ5IkVYRFhiRJqgiLDEmSVBEWGZIkqSIsMiRJUkX8f0d75OHAWPbYAAAAAElFTkSu\nQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x192cec27588>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "def sigmoid_transfer(t):\n",
    "    return 1.0 / (1+np.exp(-x))\n",
    "\n",
    "x = np.arange(-5.0, 5.0, 0.02)\n",
    "\n",
    "plt.xlabel(\"x\")\n",
    "plt.ylabel(\"y\")\n",
    "plt.plot(x, sigmoid_transfer(x), 'r')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Hyperbolic Transfer Function\n",
    "\n",
    "** Equation 1.3: The Hyperbolic Tangent Function **\n",
    "\n",
    "$$ f(x) = \\tanh(x) $$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAiUAAAF5CAYAAABAyVr6AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAAPYQAAD2EBqD+naQAAIABJREFUeJzt3XmYHVWd//H3lwAhAUmAQMIimzgIIkjCFhTQsIQ1KIIY\nQXaVn2wTZOQn6g+VUQeQdRxEYBhka1kUEsEICelACBCgwyIQQIdFdsIWlhiynd8fdQOdmHS6m+4+\nVfe+X89zn86trup8+j69fPqcU1WRUkKSJCm3ZXIHkCRJAkuJJEkqCUuJJEkqBUuJJEkqBUuJJEkq\nBUuJJEkqBUuJJEkqBUuJJEkqBUuJJEkqBUuJJEkqhcqVkojYISLGRMQLETE/Ika045gvRERLRMyK\niCcj4tCeyCpJktqvcqUEWBF4EPgOsNQb90TE+sBNwG3AFsB5wCURsWv3RZQkSR0VVb4hX0TMB76U\nUhrTxj6nA3uklDZvta0J6JdS2rMHYkqSpHao4khJR20HjF9k2y3A0AxZJEnSEjRCKRkEvLLItleA\nlSOid4Y8kiRpMZbNHaCMImI1YDjwDDArbxpJkiplBWB94JaU0usdObARSsnLwMBFtg0E3k4pvb+E\nY4YDV3VrKkmS6ttBwNUdOaARSsndwB6LbNuttn1JngG48sor2WSTTbopVn0aNWoU55xzTu4YleJr\n1jm+bh3XodfsnXfg6afhmWfgpZfghRfgxReLf7/yCix6kkT//rDaarDqqrDKKrDyyvCxj8FKKy3+\nbZ8+sMIKxaN3b4jo8s+3q/i11jHTpk3j4IMPhtrv0o6oXCmJiBWBjYAFX8EbRsQWwBsppeci4hfA\nWimlBdciuRA4pnYWzqXAzsD+QFtn3swC2GSTTRg8eHB3fBp1q1+/fr5mHeRr1jm+bh232Nds/nx4\n8kmYMgUefhgefbR4PP/8h/usuSasvz586lOw++7Fv9ddt9g+cCAMGADLVu7XSbv5tdZpHV7+UMWv\noq2AZoprlCTgrNr23wJHUCxs/fiCnVNKz0TEXsA5wPHA88CRKaVFz8iRpPo3axZMngy3314UkXvv\nhbfeKkYqNtwQPv1p+MY3irebbloUkT59cqdWg6hcKUkp3U4bZw2llA5fzLY7gCHdmUuSSmvaNPjj\nH+Huu4uplVmzYPXVYbvt4KSTYNttYeutoV+/3EnV4CpXSiRJS5FSMRVz/fXw+98XpaRv32I9x89/\nDrvuWoyElHgdhxqTpURdauTIkbkjVI6vWef4ui3Gm2/CVVfBJZfAQw8VoyIjRsDpp8OuuzLyhhvA\n163D/FrrOZW+zHx3iYjBQEtLS4uLmySV34MPwtlnw3XXwdy5sM8+cOSRsNtusNxyudOpwUydOpUh\nQ4YADEkpTe3IsY6USFIVpQTNzcUoyK23FmfE/PjHcOihMGhQ7nRSp1hKJKlq7rsPTj65KCVbbAFX\nXw0HHFDXp+WqMTTCvW8kqT48+yx89auwzTbFBcxGj4YHHijWiVhIVAcsJZJUdnPnwi9/WVw3ZPJk\nuPTS4uyaESM8g0Z1xWotSWX24INw2GHwl7/AccfBaacVl2mX6pAjJZJURinBuecWFzaD4uqr555r\nIVFds5RIUtlMnw577QWjRsExxxSFZKutcqeSup3TN5JUJgvWisycCX/6E+yx6E3OpfrlSIkklcUf\n/wif+1xxJdaWFguJGo6lRJLK4PzzYd99i/vS3HknfPzjSz9GqjOWEknK7Wc/gxNOgO9+t7iJ3oor\n5k4kZWEpkaRcUoIf/AB++EP46U/hjDNgGX8sq3G50FWSckipuFT8mWcWj5NOyp1Iys5SIkk5nH56\nUUbOPbeYupHk9I0k9biLL4bvfx9OPdVCIrViKZGknjR2LBx9dHFRtFNPzZ1GKhVLiST1lMceg699\nDfbcE847z5vpSYuwlEhST3j9ddhnH1h3XbjqKujVK3ciqXRc6CpJ3W3+fDj4YJgxA267DVZeOXci\nqZQsJZLU3U4/Hf7852I9yfrr504jlZbTN5LUnSZNKi6OdsopsPvuudNIpWYpkaTuMmMGHHQQfP7z\n8JOf5E4jlZ6lRJK6ywknwFtvweWXw7LOlktL43eJJHWH0aPht7+FSy+F9dbLnUaqBEdKJKmrvfEG\nfOtbMGIEHHZY7jRSZVhKJKmrnXwyvP8+XHihF0iTOsDpG0nqSpMmwSWXwAUXwJpr5k4jVYojJZLU\nVWbPLu5rs+228O1v504jVY4jJZLUVc4/H554AlpaYBn/5pM6yu8aSeoKr74Kp51WjJRssUXuNFIl\nWUokqSucemoxOvLjH+dOIlWW0zeS9FH95S9w0UVw1lkwYEDuNFJlOVIiSR/VySfDJz4B3/lO7iRS\npTlSIkkfxZ13Fnf/vfZaWH753GmkSnOkRJI6K6Xi7r+f/Sx85Su500iV50iJJHXWrbcWF0u76SZP\nAZa6gN9FktQZKcEPfwhDh8Kee+ZOI9UFR0okqTNuuQXuvx/Gj/f+NlIXcaREkjrjZz8rLic/bFju\nJFLdcKREkjrqjjuKs27GjHGUROpCjpRIUkf9/Oew+eaw9965k0h1xZESSeqIqVOL9SRNTY6SSF3M\nkRJJ6ohzzoH114cDDsidRKo7lhJJaq+XXoJrroHjjoNevXKnkeqOpUSS2uvXv4beveHII3MnkeqS\npUSS2mPWLLjwQjjsMOjXL3caqS5ZSiSpPZqaYPp0OP743EmkumUpkaSlSQnOOw/22gs++cncaaS6\nZSmRpKW54w546CH413/NnUSqa5YSSVqaCy+EjTeGnXfOnUSqa5YSSWrLa6/BH/4A3/ymF0uTupml\nRJLacsUVxZqSQw7JnUSqe5YSSVqSlODii+HLX4bVV8+dRqp7lhJJWpK774Zp0+Coo3InkRqCpUSS\nluTii2GDDVzgKvUQS4kkLc6MGcV9bo48EpbxR6XUE/xOk6TFaWqC2bPh8MNzJ5EahqVEkhbniitg\n+HBYa63cSaSGsWzuAJJUOk89BXfdBVdfnTuJ1FAcKZGkRV11Fay0Euy7b+4kUkOxlEhSaynBlVfC\nfvtB376500gNxVIiSa3dfz88+SQcfHDuJFLDsZRIUmtXXgmDBsGwYbmTSA2nsqUkIo6JiKcj4h8R\ncU9EbN3GvodGxPyImFd7Oz8iZvZkXkkVMGdOcSrw178OvXrlTiM1nEqWkog4EDgLOBXYEngIuCUi\nBrRx2AxgUKvHet2dU1LFjB8P06c7dSNlUslSAowCfpNSujyl9DhwNDATOKKNY1JKaXpK6dXaY3qP\nJJVUHVdfDZtsAp/9bO4kUkOqXCmJiOWAIcBtC7allBIwHhjaxqErRcQzEfH3iLgxIjbt5qiSquT9\n92HMGDjwQIjInUZqSJUrJcAAoBfwyiLbX6GYllmcJyhGUUYAB1F83ndFhJdqlFQYNw7efhv23z93\nEqlhNcQVXVNK9wD3LHgeEXcD04BvU6xLkdTorruumLr59KdzJ5EaVhVLyWvAPGDgItsHAi+35wOk\nlOZGxAPARm3tN2rUKPr167fQtpEjRzJy5Mj2p5VUfrNnw+jRcMIJuZNIldLU1ERTU9NC22bMmNHp\njxfFcoxqiYh7gCkppRNqzwP4O3B+SunMdhy/DPAocHNK6aTFvH8w0NLS0sLgwYO7Nryk8vnTn2Cv\nveDhh+Ezn8mdRqq0qVOnMmTIEIAhKaWpHTm2iiMlAGcDl0VEC3Avxdk4fYHLACLicuD5lNIptec/\nopi++RvQH/gesC5wSY8nl1Q+118PG28Mm22WO4nU0CpZSlJK19auSfJTimmbB4HhrU7zXQeY2+qQ\nVYCLKBbCvgm0AENrpxNLamRz5sCNN8Ixx3jWjZRZJUsJQErpAuCCJbxv2CLPTwRO7Ilckirmttvg\nzTc960YqgSqeEixJXef66+GTn4TNN8+dRGp4lhJJjWvevOKCafvt59SNVAKWEkmNa8qU4l43++6b\nO4kkLCWSGtno0bDGGrDNNrmTSMJSIqmRjR4N++wDvXrlTiIJS4mkRvXEE8VjxIjcSSTVWEokNaYx\nY6BPH9hll9xJJNVYSiQ1pjFjYNddoW/f3Ekk1VhKJDWe6dPhrrs860YqGUuJpMZz882QUnETPkml\nYSmR1HhGj4ahQ2HgwNxJJLViKZHUWN5/H8aNg733zp1E0iIsJZIay513wnvvwR575E4iaRGWEkmN\nZexYWHNN2GKL3EkkLcJSIqmxjB1bjJJ4Az6pdCwlkhrHs8/CY485dSOVlKVEUuMYO7a4z41XcZVK\nyVIiqXGMHQvbbw/9++dOImkxLCWSGsP778Nttzl1I5WYpURSY5g0qTgVeM89cyeRtASWEkmNYexY\nWGst2Hzz3EkkLYGlRFJjGDsWdt/dU4GlErOUSKp/zz4L06a5nkQqOUuJpPrnqcBSJVhKJNU/TwWW\nKsFSIqm+zZkDzc2w2265k0haCkuJpPp2773wzjuw6665k0haCkuJpPo2fnwxbbPVVrmTSFoKS4mk\n+jZuHAwbVix0lVRqlhJJ9evtt+Gee5y6kSrCUiKpfk2cCPPmWUqkirCUSKpf48bBBhvAJz6RO4mk\ndrCUSKpf48Y5SiJViKVEUn167jl44gmv4ipViKVEUn0aP764+d6wYbmTSGonS4mk+jRuHAwZAqut\nljuJpHaylEiqP/PnFyMlrieRKsVSIqn+PPwwTJ9uKZEqxlIiqf6MHw99+xZ3BpZUGZYSSfVnwgT4\n/Oehd+/cSSR1gKVEUn2ZMwcmTYIvfjF3EkkdZCmRVF9aWuDddy0lUgVZSiTVl+ZmWGklGDw4dxJJ\nHWQpkVRfmpthhx1gueVyJ5HUQZYSSfVj9myYPNmpG6miLCWS6sd998HMmZYSqaIsJZLqR3MzrLwy\nbLll7iSSOsFSIql+NDfDjjtCr165k0jqBEuJpPrw/vtw111O3UgVZimRVB+mTIFZsywlUoVZSiTV\nh+ZmWGUV2GKL3EkkdZKlRFJ9WLCeZBl/rElV5XevpOr7xz/g7rudupEqzlIiqfruuae4cJqlRKo0\nS4mk6mtuhtVWg802y51E0kdgKZFUfc3NsNNOrieRKs7vYEnVNnNmcTqwUzdS5VlKJFXb5MkwZ46l\nRKoDlhJJ1TZxIqyxBmy6ae4kkj4iS4mkamtuhi98ASJyJ5H0EVlKJFXXu+/CffcVpURS5VlKJFXX\nnXfC3LmuJ5HqhKVEUnVNnAiDBsHGG+dOIqkLWEokVVdzczFK4noSqS5YSiRV09tvQ0uL60mkOmIp\nkVRNkybBvHmuJ5HqiKVEUjU1N8Paa8NGG+VOIqmLVLaURMQxEfF0RPwjIu6JiK2Xsv8BETGttv9D\nEbFHT2WV1A0mTnQ9iVRnKllKIuJA4CzgVGBL4CHglogYsIT9tweuBi4GPguMBm6MCC8BKVXRW2/B\nAw+4nkSqM5UsJcAo4DcppctTSo8DRwMzgSOWsP/xwNiU0tkppSdSSv8PmAoc2zNxJXWpO+6A+fNd\nTyLVmQ6Xkoj4bUTs2B1h2vn/LwcMAW5bsC2llIDxwNAlHDa09v7Wbmljf0ll1twM664LG2yQO4mk\nLtSZkZJ+wPiI+GtEnBIRa3d1qKUYAPQCXllk+yvAoCUcM6iD+0sqM69PItWlDpeSlNKXgLWBXwMH\nAs9ExNiI2L82iiFJ3eeNN+Dhh11PItWhZTtzUEppOnA2cHZEDAYOB64A3o2IK4ELUkp/7bqYC3kN\nmAcMXGT7QODlJRzzcgf3B2DUqFH069dvoW0jR45k5MiR7Q4rqYvdfjuk5HoSqQSamppoampaaNuM\nGTM6/fGiWI7RyYMj1gQOoSgl6wC/pxhF2Qn4XkrpnE5/8Lb/33uAKSmlE2rPA/g7cH5K6czF7P87\noE9Kad9W2yYDD6WUvrOY/QcDLS0tLQwePLg7PgVJnXX88XDTTfDUU7mTSFqMqVOnMmTIEIAhKaWp\nHTm2wyMltSmaERRFZDfgYeBc4OqU0tu1fUYCFwDdUkooRmkui4gW4F6Ks3H6ApfV/v/LgedTSqfU\n9j8PmBgRJwI3AyMpFst+s5vySeouC9aTSKo7nZm+eYliLUoTsE1K6cHF7PNn4K2PEqwtKaVra9ck\n+SnFNMyDwPDatBIUozZzW+1/d0R8HfhZ7fFXYN+U0mPdlVFSN5g+HR55BE4+OXcSSd2gM6VkFHBd\nSmnWknZIKb0JdOu5eimlCyhGYxb3vmGL2fZ7iuklSVV1++3FWxe5SnWpw6UkpXRFdwSRpKVqbi7u\ndbPOOrmTSOoGVb2iq6RG5HoSqa5ZSiRVw8svw7RplhKpjllKJFWD60mkumcpkVQNzc2w8caw5pq5\nk0jqJpYSSdXgehKp7llKJJXfiy/Ck09aSqQ6ZymRVH7NzcXbnXbKm0NSt7KUSCq/5mbYbDMYuOh9\nNSXVE0uJpPKbMMGpG6kBWEokldszz8DTT8Owf7p7hKQ6YymRVG7NzRDhehKpAVhKJJXbhAmw5Zaw\nyiq5k0jqZpYSSeWVktcnkRqIpURSef31r/DCC64nkRqEpURSeTU3Q69esMMOuZNI6gGWEknlNWEC\nbL01fOxjuZNI6gGWEknl5HoSqeFYSiSV06OPwvTprieRGoilRFI5NTfD8svD9tvnTiKph1hKJJXT\nhAmw3XbQt2/uJJJ6iKVEUvnMmwe33+7UjdRgLCWSyuehh+DNN13kKjUYS4mk8pkwAfr0gW23zZ1E\nUg+ylEgqn+Zm+NznoHfv3Ekk9SBLiaRymTMH7rjD9SRSA7KUSCqXlhZ4913Xk0gNyFIiqVwmTCgu\nK7/VVrmTSOphlhJJ5XLbbbDjjrDssrmTSOphlhJJ5TFzJtx5J+y6a+4kkjKwlEgqj0mTYPZsS4nU\noCwlkspj3DhYay3YZJPcSSRlYCmRVB7jxhWjJBG5k0jKwFIiqRxeeQUefhh22SV3EkmZWEoklcP4\n8cVbS4nUsCwlksph/Hj4zGdg0KDcSSRlYimRlF9KH64nkdSwLCWS8nv8cXjhBUuJ1OAsJZLyGzcO\nll8edtghdxJJGVlKJOU3bhxsvz2suGLuJJIyspRIymvOHJg40akbSZYSSZndcw+8+66lRJKlRFJm\nt94Kq6wCgwfnTiIpM0uJpLzGjoXhw6FXr9xJJGVmKZGUzyuvQEsL7LFH7iSSSsBSIimfW24p3g4f\nnjeHpFKwlEjKZ+xYGDIEBg7MnURSCVhKJOUxb14xUuLUjaQaS4mkPKZMgTffhD33zJ1EUklYSiTl\nMXYsrLoqbLNN7iSSSsJSIimPsWNht908FVjSBywlknqepwJLWgxLiaSet+BU4N13z5tDUqlYSiT1\nvLFjYautYI01cieRVCKWEkk9a+5cTwWWtFiWEkk96957i1OBLSWSFmEpkdSzbroJBgzwVGBJ/8RS\nIqlnjR4Ne+/tqcCS/omlRFLP+dvf4LHHYMSI3EkklZClRFLPGTMGevcuLpomSYuwlEjqOWPGwC67\nwIor5k4iqYQsJZJ6xuuvw6RJsO++uZNIKilLiaSe8ac/wfz5xSJXSVoMS4mknjF6NGy7Lay5Zu4k\nkkrKUiKp+82aBX/+s1M3ktpkKZHU/Zqb4b33PBVYUpssJZK63x/+ABttBJtumjuJpBKrXCmJiFUi\n4qqImBERb0bEJRHR5vmFETExIua3esyLiAt6KrPU0ObMgRtugP33h4jcaSSV2LK5A3TC1cBAYGdg\neeAy4DfAwW0ck4CLgB8BC34qzuy+iJI+cPvtxenABxyQO4mkkqtUKYmITwHDgSEppQdq244Dbo6I\nk1JKL7dx+MyU0vSeyCmpleuugw03hC23zJ1EUslVbfpmKPDmgkJSM55iJGTbpRx7UERMj4i/RMTP\nI6JPt6WUVJg7t1hP4tSNpHao1EgJMAh4tfWGlNK8iHij9r4luQp4FngR2Bw4A/gXYP9uyikJ4I47\n4LXXnLqR1C6lKCUR8Qvg5DZ2ScAmnf34KaVLWj19NCJeBsZHxAYppaeXdNyoUaPo16/fQttGjhzJ\nyJEjOxtFaizXXQfrrw9DhuROIqkbNDU10dTUtNC2GTNmdPrjRUrpo2b6yCJiNWC1pez2FPAN4Jcp\npQ/2jYhewCxg/5TS6Hb+f32Bd4HhKaVxi3n/YKClpaWFwYMHt/OzkLSQefNgrbXgkEPgzDNzp5HU\nQ6ZOncqQ4g+RISmlqR05thQjJSml14HXl7ZfRNwN9I+ILVutK9mZ4oyaKR34L7ekGH15qaNZJbXT\npEnw6qvFehJJaodKLXRNKT0O3AJcHBFbR8TngP8EmhaceRMRa0XEtIjYqvZ8w4j4YUQMjoj1ImIE\n8Fvg9pTSI7k+F6nuXXMNrLsubLNN7iSSKqJSpaTm68DjFGfd3ATcAXy71fuXo1jE2rf2fDawC0WZ\nmQacCVwHeL1rqbvMng3XXgsjR3rWjaR2K8X0TUeklN6ijQulpZSeBXq1ev488IXuTybpA3/+M7zx\nBhzc1jUNJWlhVRwpkVR2V14JW2wBm22WO4mkCrGUSOpaM2bAmDGOkkjqMEuJpK71+98Xa0q8no+k\nDrKUSOpaV14Jw4bB2mvnTiKpYiwlkrrOc8/BxIlO3UjqFEuJpK7T1AS9e8N+++VOIqmCLCWSukZK\n8D//A1/6Eqy8cu40kirIUiKpa0yeDI8/DkcdlTuJpIqylEjqGhdfDBtuCF/8Yu4kkirKUiLpo3vr\nLbjuOjjySFjGHyuSOsefHpI+uquvLq5NcvjhuZNIqjBLiaSPJqVi6mbvvWHNNXOnkVRhlhJJH01L\nCzz4IHzzm7mTSKo4S4mkj+bii4urtw4fnjuJpIqzlEjqvLfeKi4rf9RRsOyyudNIqjhLiaTOu/RS\nmDMHjj46dxJJdcBSIqlz5s2DX/0KDjwQBg3KnUZSHXC8VVLn3HwzPP00XHNN7iSS6oQjJZI65/zz\nYbvtYOutcyeRVCccKZHUcQ8/DLfdVlw0TZK6iCMlkjru9NNhvfXggANyJ5FURxwpkdQxzzxTrCM5\n5xxPA5bUpRwpkdQxZ50F/fvDEUfkTiKpzlhKJLXf9Onw3/8Nxx0HK66YO42kOmMpkdR+554LEXDs\nsbmTSKpDlhJJ7fPaa3DeecUoyWqr5U4jqQ5ZSiS1z5lnFqMkJ52UO4mkOmUpkbR0r75aXFL++ONh\nwIDcaSTVKUuJpKU7/XTo1Qu++93cSSTVMUuJpLY980wxSnLiibDqqrnTSKpjlhJJbfvBD4oy4loS\nSd3MyzFKWrL77y/ub3PRRbDSSrnTSKpzjpRIWryU4N/+DTbdFA4/PHcaSQ3AkRJJi3f99TBxItx8\ns/e4kdQjHCmR9M/eeQdGjYIRI2DPPXOnkdQgLCWS/tlpp8EbbxRXcJWkHmIpkbSwRx+Fc84pzrpZ\nf/3caSQ1EEuJpA/NnQtHHAGf+ISnAEvqca5ek/Shc84pTgOePBl6986dRlKDcaREUuHxx+FHPyoW\nuG63Xe40khqQpUQSzJ4NhxwC665bLHKVpAycvpFUjJA88ADcdRf06ZM7jaQGZSmRGt2tt8IZZxSP\nrbfOnUZSA3P6RmpkL7xQTNsMHw7f/W7uNJIanKVEalSzZsGXvwzLLQeXXw7L+ONAUl5O30iNKCU4\n+mj4y1/gzjthjTVyJ5IkS4nUkH75S/jtb+GKK2DIkNxpJAlw+kZqPFdfDd/7HpxyChx8cO40kvQB\nS4nUSG67DQ47DA49FP7933OnkaSFWEqkRnHHHTBiBOy8M1x8MUTkTiRJC7GUSI3gzjthzz1h6FD4\nwx+KM24kqWQsJVK9Gz8e9tgDttkGxozxiq2SSstSItWza68tRkh22AH++Efo2zd3IklaIkuJVI9S\nKi4b/7WvwYEHwujRsOKKuVNJUpssJVK9mTkTvv51OPlk+P73i+uRuIZEUgV48TSpnjz1FHzlK/Dk\nk8XUzQEH5E4kSe3mSIlUD1KCSy+FLbaAt9+Gu++2kEiqHEuJVHWvvgr77QdHHglf/So88ABsvnnu\nVJLUYU7fSFU1bx785jfwgx/AsssW1x/58pdzp5KkTnOkRKqiyZOL644ccwzsvz9Mm2YhkVR5lhKp\nSqZOLa478vnPF8/vvru4ZPyAAXlzSVIXsJRIZZcSTJoEX/oSDBlSnGFzzTVw332w3Xa500lSl7GU\nSGX1j3/AVVfBttvCjjvCE08UZ9g88kixoHUZv30l1RcXukplkhJMmQKXXQa/+x3MmAHDhsHNN8Pu\nu1tEJNU1S4mU27x5xdqQG28sHv/7v7DOOnDssXDoofDJT+ZOKEk9wlIi9bSUinUhEydCczPceitM\nnw4DB8KIEXDBBbDzztCrV+6kktSjKjUWHBGnRMTkiHgvIt7owHE/jYgXI2JmRIyLiI26M2cja2pq\nyh2hfGbOLEZCfvUrOOQQWG892Ggj+Na34PHHadp2W7jrLnjxRbjoIthtNwtJO/i11nG+Zp3j69Zz\nKlVKgOWAa4Fft/eAiDgZOBb4FrAN8B5wS0Qs3y0JG1xDf/POng2PPQY33AC/+EUx9bL55rDyyrD9\n9nDiicX7998fxoyB11+H+++nKQKGDnW9SAc19NdaJ/madY6vW8+p1PRNSuknABFxaAcOOwE4LaV0\nU+3YQ4BXgC9RFBxp6VIqFp2+/DI89xz8/e/F2wX/fuYZePrpYn0IQP/+sPHGRdk49tjiVN7NNoPe\nvbN+GpJUZpUqJR0VERsAg4DbFmxLKb0dEVOAoVhKGsvcufDOO8UN65b09u23i/Ud06fDa68t/O85\ncxb+eGuuCR//OKy7brEWZOON4VOfKh6rrw4ReT5PSaqoui4lFIUkUYyMtPZK7X3V88gj8N57xV/u\nULxd8Fj0eXv26ejzpe3z4otw3XVLfv/cucVoQuu3i9vWnve9/37xmDXrw0dbzxctFYvq27eYahkw\noCgVq69eFIzWzwcOLErI2mvD8s4ASlJXyl5KIuIXwMlt7JKATVJKT/ZQJIAVAKZNm9aD/2U7HXgg\n/O1vuVMs0Qxg6le/2r6de/Uq1lH06vXhY5llipvLLW576+e9esFyyxXTIcsvXzz691/4ee/eC++z\nwgqw4orHT+I+AAAG/UlEQVRF+VhppeLtiisWjz59iv+3Pd58s3h0kRkzZjB16tQu+3iNwtet43zN\nOsfXrWNa/e5coaPHRlrwl2wmEbEasNpSdnsqpTS31TGHAueklFZdysfeAPhf4LMppYdbbZ8IPJBS\nGrWE474OXNW+z0CSJC3GQSmlqztyQPaRkpTS68Dr3fSxn46Il4GdgYcBImJlYFvgv9o49BbgIOAZ\nYFZ3ZJMkqU6tAKxP8bu0Q7KXko6IiI8DqwLrAb0iYovau/6WUnqvts/jwMkppdG1950L/DAi/kZR\nMk4DngdGswS1otShdidJkj5wV2cOqlQpAX4KHNLq+YJJvi8Cd9T+/Umg34IdUkpnRERf4DdAf2AS\nsEdKaXb3x5UkSe2VfU2JJEkSVO+KrpIkqU5ZSiRJUilYStohIvaKiHtqN/R7IyL+kDtTVUTE8hHx\nYETMj4jNc+cpq4hYLyIuiYinal9nf42IH0fEcrmzlU1EHBMRT0fEP2rfl1vnzlRmEfH9iLg3It6O\niFci4oaI+JfcuaokIv5v7WfY2bmzlF1ErBURV0TEa7WfZQ9FxOD2Hm8pWYqI+ApwOfDfwGeA7fHM\nnI44g+JsJxcvte1TQADfBDYFRgFHAz/LGapsIuJA4CzgVGBL4CGKG2wOyBqs3HYA/pPiUgi7UNzY\n9NaI6JM1VUXUSu+3KL7W1IaI6A9MBt4HhgObAN8F2n21SRe6tiEielGcRvyjlNJledNUT0TsAfwS\n+ArwGItcxE5ti4iTgKNTShvlzlIWEXEPMCWldELteQDPAeenlM7IGq4iagXuVWDHlNKdufOUWUSs\nBLQA/wf4EcVFN0/Mm6q8IuI/gKEppZ06+zEcKWnbYGAtgIiYGhEvRsSfIuLTmXOVXkQMBC4CDgb+\nkTlOVfUH3sgdoixqU1lDWPgGmwkYT3GDTbVPf4qRS7+2lu6/gD+mlCbkDlIR+wD3R8S1tanCqRFx\nVEc+gKWkbRtSDKmfSnGNlL0ohqEm1oaptGT/A1yQUnogd5AqioiNgGOBC3NnKZEBQC/q6QabPaw2\nsnQucGdK6bHcecosIr4GfBb4fu4sFbIhxajSE8BuwK+B8yPiG+39AA1ZSiLiF7VFS0t6zKstBFvw\n+vx7SunG2i/Ywyn+yjgg2yeQSXtft4g4HlgJOH3BoRljZ9WBr7XWx6wNjAWuSSldmie56tQFFGuW\nvpY7SJlFxDoU5e2glNJSbi+uVpYBWlJKP0opPZRSuhi4mGJ9XLtU7YquXeWXFH/Jt+UpalM3wAe3\nPEwpzY6Ip4B1uylbmbXndXua4gq7Q4H3iz/MPnB/RFyVUjq8m/KVUXu/1oBi5TowgeIv2W93Z7AK\neg2YBwxcZPtA4OWej1MtEfErYE9gh5TSS7nzlNwQYHVganz4Q6wXsGNEHAv0Ti7IXJyXaPX7smYa\nsF97P0BDlpL23gQwIlooVhFvTO06/rV57fWBZ7sxYil14HU7DvhBq01rUdyY6avAvd2Trpw6csPJ\n2gjJBOA+4IjuzFVFKaU5te/JnYEx8MF0xM7A+TmzlV2tkOwL7JRS+nvuPBUwnuJsy9Yuo/gF+x8W\nkiWaTPH7srWN6cDvy4YsJe2VUnonIi4EfhIRz1O8sN+jmL65Lmu4EkspPd/6eUS8RzGF81RK6cU8\nqcqtNkIykWKk6XvAGgv+QEspLbqGopGdDVxWKyf3Upw63ZfiF4YWIyIuAEYCI4D3aovQAWaklLwL\n+mLUbvC60Jqb2s+x11NKi44E6EPnAJMj4vvAtRSnoR9FcamDdrGULN1JwByKa5X0AaYAw1JKM7Km\nqh7/smjbrhSLxDakOMUViiKXKIaNBaSUrq2d0vpTimmbB4HhKaXpeZOV2tEUX0cTF9l+OMXPNbWP\nP8OWIqV0f0R8GfgPilOonwZOSCn9rr0fw+uUSJKkUmjIs28kSVL5WEokSVIpWEokSVIpWEokSVIp\nWEokSVIpWEokSVIpWEokSVIpWEokSVIpWEokSVIpWEokSVIpWEokSVIpWEokVUJEDIiIlyLi/7ba\ntn1EvB8RX8yZTVLX8IZ8kiojIvYAbgSGAk9S3CX4hpTSv2UNJqlLWEokVUpE/CewK3A/sBmwdUpp\nTt5UkrqCpURSpUTECsAjwDrA4JTSY5kjSeoirimRVDUbAWtR/PzaIHMWSV3IkRJJlRERywH3Ag8A\nTwCjgM1SSq9lDSapS1hKJFVGRJwJ7AdsDswEJgJvp5T2yZlLUtdw+kZSJUTETsDxwMEppfdS8RfV\nIcDnI+LbedNJ6gqOlEiSpFJwpESSJJWCpUSSJJWCpUSSJJWCpUSSJJWCpUSSJJWCpUSSJJWCpUSS\nJJWCpUSSJJWCpUSSJJWCpUSSJJWCpUSSJJWCpUSSJJXC/wd1LyTLEvD5AQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x192cda7f860>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "def tanh_transfer(t):\n",
    "    return np.tanh(t)\n",
    "\n",
    "x = np.arange(-5.0, 5.0, 0.02)\n",
    "\n",
    "plt.xlabel(\"x\")\n",
    "plt.ylabel(\"y\")\n",
    "plt.plot(x, tanh_transfer(x), 'r')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$ f(x) = \\frac{1}{1 + e^{-x}} $$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Calculating a Neuron\n",
    "\n",
    "Equation 1.2: Calculate H1\n",
    "$$ h_1 = A(\\sum_{c=1}^n (i_c * w_c)) $$\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "\n",
    "a = np.array( [1, 1, 0] ) # First 1 is the bias, 1 and 0 are the inputs\n"
   ]
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python [conda env:tensorflow]",
   "language": "python",
   "name": "conda-env-tensorflow-py"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}
